Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j1xSG-0004fG-Uc for pgsql-hackers@arkaria.postgresql.org; Wed, 12 Feb 2020 19:13:49 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1j1xSD-00031J-Fh for pgsql-hackers@arkaria.postgresql.org; Wed, 12 Feb 2020 19:13:45 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j1xSC-00031C-WA for pgsql-hackers@lists.postgresql.org; Wed, 12 Feb 2020 19:13:45 +0000 Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1j1xS3-0004EO-2f for pgsql-hackers@postgresql.org; Wed, 12 Feb 2020 19:13:44 +0000 Received: by mail-lj1-x229.google.com with SMTP id o15so3635688ljg.6 for ; Wed, 12 Feb 2020 11:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EsojdX5skfGdtwwRX2gACFx6SzrzNmdqrxzXmnjF89k=; b=CBY8PuKdocdUI4IgnCKvPtkMzQq0fr0z2BnR9Cvi5UEHRdrWGwlBwOXT7l+McRP1j8 W8fcmpb2oPlyWWdIUcAh4a9AU9CopB86z27ZrWTIha/YTlFhrTu2EI8SrUmmaFCtLNFP T/EfWJd6NE6fdBC7apAs/3nrsY7IcaGGCpPSnPs4S84y/JSlYhpBhhgQCY1/f2S1u9TQ ks4iYW4q1LLhv8DJa4+B+h5bPLXYG0NrRSUfmkOaE65wLNmClEL1hZ9p3xkIncWUGICz khRJFUxgvUModCIwt7jTl2T2MP7ew/VFzk7jDGIfpb1hhJAxnM7KvoUoy3/xqu6BAjXo 3tiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EsojdX5skfGdtwwRX2gACFx6SzrzNmdqrxzXmnjF89k=; b=Gsu1I/WZNsjpC2COivivlsduwVX0rVJYnukxPveq7CcMZzXhRUemIpaxdRvtVvGws1 SxMGKpFhDOCK3vfE/JVgEcX19Q7Zahx8V2hOEqeDVX7VOUaO7NvDUYS/xjZ2m252OUkR jNBRhvcsAuHqtdIhD0MuYx8+HD5mEFqxwFpCfqeYKzJAxQeYBiI+u2S43P89T8W0eDBc EVBIrvXqromUwy9J/chc/IypY1+BJFUcgX4ozOnIEHnoVo9Lv9Zukd8PTtsAHix48Wvd hg8Ooe8lCw8mKjvCGPJBU3OQLR/PODH42VbGaV4UjaBGTq1nIBhaZU+nFO1q4MzbWdWd h5cg== X-Gm-Message-State: APjAAAWF/cXGWpOAoKSB+eDjvTjJ6EeeXpe4HZYiZFn4M8F/BOV9/i6H RH7bz7Q559rd9xLeAlnJDqk= X-Google-Smtp-Source: APXvYqxM4tqGIrMcJ0K61oCxPgYtF+9PWtLfxu5MTZZW58MiwF2FxrgP4PBD3chkBPMPlEZy2Kiq7w== X-Received: by 2002:a2e:8490:: with SMTP id b16mr8769810ljh.282.1581534812915; Wed, 12 Feb 2020 11:13:32 -0800 (PST) Received: from nol (82-64-124-11.subs.proxad.net. [82.64.124.11]) by smtp.gmail.com with ESMTPSA id v9sm48216lfe.18.2020.02.12.11.13.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2020 11:13:30 -0800 (PST) Date: Wed, 12 Feb 2020 20:13:26 +0100 From: Julien Rouhaud To: Peter Eisentraut Cc: Thomas Munro , Robert Haas , Michael Paquier , Laurenz Albe , Douglas Doole , Christoph Berg , Pg Hackers Subject: Re: Collation versioning Message-ID: <20200212191326.GA72685@nol> References: <3ede40e7-5799-7096-dc5f-d7beda8e7145@2ndquadrant.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="6c2NcOVqGQ03X4Wi" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote: > > Note that I didn't change any syntax (or switched to native functions > for the binary pg_dump) as it's still not clear to me what exactly > should be implemented. Hearing no complaints on the suggestions, I'm attaching v8 to address that: - pg_dump is now using a binary_upgrade_set_index_coll_version() function rather than plain DDL - the additional DDL is now of the form: ALTER INDEX name ALTER COLLATION name REFRESH VERSION I also added an alternate file for the collate.icu.utf8, so the build farm bot should turn green for the linux part. --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0001-Remove-pg_collation.collversion-v8.patch" From dd68cfb8b213f83536e482ed6c7b6170cb155467 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 28 May 2019 14:15:28 -0400 Subject: [PATCH 1/6] Remove pg_collation.collversion. A later patch will add version tracking for individual database objects that depend on collations. Author: Thomas Munro Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- doc/src/sgml/catalogs.sgml | 11 --- doc/src/sgml/func.sgml | 5 +- doc/src/sgml/ref/alter_collation.sgml | 63 -------------- src/backend/catalog/pg_collation.c | 5 -- src/backend/commands/collationcmds.c | 85 ------------------- src/backend/nodes/copyfuncs.c | 13 --- src/backend/nodes/equalfuncs.c | 11 --- src/backend/parser/gram.y | 18 +--- src/backend/tcop/utility.c | 12 --- src/backend/utils/adt/pg_locale.c | 37 -------- src/bin/pg_dump/pg_dump.c | 8 +- src/include/catalog/pg_collation.dat | 7 +- src/include/catalog/pg_collation.h | 5 -- src/include/catalog/toasting.h | 1 - src/include/commands/collationcmds.h | 1 - src/include/nodes/parsenodes.h | 11 --- .../regress/expected/collate.icu.utf8.out | 3 - src/test/regress/sql/collate.icu.utf8.sql | 5 -- 18 files changed, 11 insertions(+), 290 deletions(-) diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index a10b66569b..f187a1af65 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<iteration count>:&l LC_CTYPE for this collation object - - - collversion - text - - - Provider-specific version of the collation. This is recorded when the - collation is created and then checked when it is used, to detect - changes in the collation definition that could lead to data corruption. - - diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index ceda48e0fc..44d82ca0b3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21103,10 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); pg_collation_actual_version returns the actual version of the collation object as it is currently installed in the - operating system. If this is different from the value - in pg_collation.collversion, then objects depending on - the collation might need to be rebuilt. See also - . + operating system. diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml index 4cfcb42251..4241ec9f5a 100644 --- a/doc/src/sgml/ref/alter_collation.sgml +++ b/doc/src/sgml/ref/alter_collation.sgml @@ -88,72 +88,9 @@ ALTER COLLATION name SET SCHEMA new_sche - - REFRESH VERSION - - - Update the collation's version. - See below. - - - - - Notes - - - When using collations provided by the ICU library, the ICU-specific version - of the collator is recorded in the system catalog when the collation object - is created. When the collation is used, the current version is - checked against the recorded version, and a warning is issued when there is - a mismatch, for example: - -WARNING: collation "xx-x-icu" has version mismatch -DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5. -HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version. - - A change in collation definitions can lead to corrupt indexes and other - problems because the database system relies on stored objects having a - certain sort order. Generally, this should be avoided, but it can happen - in legitimate circumstances, such as when - using pg_upgrade to upgrade to server binaries linked - with a newer version of ICU. When this happens, all objects depending on - the collation should be rebuilt, for example, - using REINDEX. When that is done, the collation version - can be refreshed using the command ALTER COLLATION ... REFRESH - VERSION. This will update the system catalog to record the - current collator version and will make the warning go away. Note that this - does not actually check whether all affected objects have been rebuilt - correctly. - - - When using collations provided by libc and - PostgreSQL was built with the GNU C library, the - C library's version is used as a collation version. Since collation - definitions typically change only with GNU C library releases, this provides - some defense against corruption, but it is not completely reliable. - - - Currently, there is no version tracking for the database default collation. - - - - The following query can be used to identify all collations in the current - database that need to be refreshed and the objects that depend on them: - pg_collation_actual_version(c.oid) - ORDER BY 1, 2; -]]> - - - Examples diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c index 8559779a4f..c78192e34b 100644 --- a/src/backend/catalog/pg_collation.c +++ b/src/backend/catalog/pg_collation.c @@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace, bool collisdeterministic, int32 collencoding, const char *collcollate, const char *collctype, - const char *collversion, bool if_not_exists, bool quiet) { @@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace, values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate); namestrcpy(&name_ctype, collctype); values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype); - if (collversion) - values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion); - else - nulls[Anum_pg_collation_collversion - 1] = true; tup = heap_form_tuple(tupDesc, values, nulls); diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c index 85f726ae06..493aa21a14 100644 --- a/src/backend/commands/collationcmds.c +++ b/src/backend/commands/collationcmds.c @@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e bool collisdeterministic = true; int collencoding = 0; char collprovider = 0; - char *collversion = NULL; Oid newoid; ObjectAddress address; @@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e if (deterministicEl) collisdeterministic = defGetBoolean(deterministicEl); - if (versionEl) - collversion = defGetString(versionEl); - if (collproviderstr) { if (pg_strcasecmp(collproviderstr, "icu") == 0) @@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e } } - if (!collversion) - collversion = get_collation_actual_version(collprovider, collcollate); - newoid = CollationCreate(collName, collNamespace, GetUserId(), @@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e collencoding, collcollate, collctype, - collversion, if_not_exists, false); /* not quiet */ @@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid) collname, get_namespace_name(nspOid)))); } -/* - * ALTER COLLATION - */ -ObjectAddress -AlterCollation(AlterCollationStmt *stmt) -{ - Relation rel; - Oid collOid; - HeapTuple tup; - Form_pg_collation collForm; - Datum collversion; - bool isnull; - char *oldversion; - char *newversion; - ObjectAddress address; - - rel = table_open(CollationRelationId, RowExclusiveLock); - collOid = get_collation_oid(stmt->collname, false); - - if (!pg_collation_ownercheck(collOid, GetUserId())) - aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION, - NameListToString(stmt->collname)); - - tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid)); - if (!HeapTupleIsValid(tup)) - elog(ERROR, "cache lookup failed for collation %u", collOid); - - collForm = (Form_pg_collation) GETSTRUCT(tup); - collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion, - &isnull); - oldversion = isnull ? NULL : TextDatumGetCString(collversion); - - newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate)); - - /* cannot change from NULL to non-NULL or vice versa */ - if ((!oldversion && newversion) || (oldversion && !newversion)) - elog(ERROR, "invalid collation version change"); - else if (oldversion && newversion && strcmp(newversion, oldversion) != 0) - { - bool nulls[Natts_pg_collation]; - bool replaces[Natts_pg_collation]; - Datum values[Natts_pg_collation]; - - ereport(NOTICE, - (errmsg("changing version from %s to %s", - oldversion, newversion))); - - memset(values, 0, sizeof(values)); - memset(nulls, false, sizeof(nulls)); - memset(replaces, false, sizeof(replaces)); - - values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion); - replaces[Anum_pg_collation_collversion - 1] = true; - - tup = heap_modify_tuple(tup, RelationGetDescr(rel), - values, nulls, replaces); - } - else - ereport(NOTICE, - (errmsg("version has not changed"))); - - CatalogTupleUpdate(rel, &tup->t_self, tup); - - InvokeObjectPostAlterHook(CollationRelationId, collOid, 0); - - ObjectAddressSet(address, CollationRelationId, collOid); - - heap_freetuple(tup); - table_close(rel, NoLock); - - return address; -} - - Datum pg_collation_actual_version(PG_FUNCTION_ARGS) { @@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS) collid = CollationCreate(localebuf, nspid, GetUserId(), COLLPROVIDER_LIBC, true, enc, localebuf, localebuf, - get_collation_actual_version(COLLPROVIDER_LIBC, localebuf), true, true); if (OidIsValid(collid)) { @@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS) collid = CollationCreate(alias, nspid, GetUserId(), COLLPROVIDER_LIBC, true, enc, locale, locale, - get_collation_actual_version(COLLPROVIDER_LIBC, locale), true, true); if (OidIsValid(collid)) { @@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS) nspid, GetUserId(), COLLPROVIDER_ICU, true, -1, collcollate, collcollate, - get_collation_actual_version(COLLPROVIDER_ICU, collcollate), true, true); if (OidIsValid(collid)) { diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index 54ad62bb7f..bf4f793ba2 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from) return newnode; } -static AlterCollationStmt * -_copyAlterCollationStmt(const AlterCollationStmt *from) -{ - AlterCollationStmt *newnode = makeNode(AlterCollationStmt); - - COPY_NODE_FIELD(collname); - - return newnode; -} - static AlterDomainStmt * _copyAlterDomainStmt(const AlterDomainStmt *from) { @@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from) case T_AlterTableCmd: retval = _copyAlterTableCmd(from); break; - case T_AlterCollationStmt: - retval = _copyAlterCollationStmt(from); - break; case T_AlterDomainStmt: retval = _copyAlterDomainStmt(from); break; diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index 5b1ba143b1..4ec777a78c 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b) return true; } -static bool -_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b) -{ - COMPARE_NODE_FIELD(collname); - - return true; -} - static bool _equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b) { @@ -3269,9 +3261,6 @@ equal(const void *a, const void *b) case T_AlterTableCmd: retval = _equalAlterTableCmd(a, b); break; - case T_AlterCollationStmt: - retval = _equalAlterCollationStmt(a, b); - break; case T_AlterDomainStmt: retval = _equalAlterDomainStmt(a, b); break; diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 1b0edf5d3d..b6d6a0e239 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); } %type stmt schema_stmt - AlterEventTrigStmt AlterCollationStmt + AlterEventTrigStmt AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt AlterFdwStmt AlterForeignServerStmt AlterGroupStmt AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt @@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt stmt : AlterEventTrigStmt - | AlterCollationStmt | AlterDatabaseStmt | AlterDatabaseSetStmt | AlterDefaultPrivilegesStmt @@ -10321,21 +10320,6 @@ drop_option: } ; -/***************************************************************************** - * - * ALTER COLLATION - * - *****************************************************************************/ - -AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P - { - AlterCollationStmt *n = makeNode(AlterCollationStmt); - n->collname = $3; - $$ = (Node *)n; - } - ; - - /***************************************************************************** * * ALTER SYSTEM diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index bb85b5e52a..e57e05b9ee 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate, address = AlterStatistics((AlterStatsStmt *) parsetree); break; - case T_AlterCollationStmt: - address = AlterCollation((AlterCollationStmt *) parsetree); - break; - default: elog(ERROR, "unrecognized node type: %d", (int) nodeTag(parsetree)); @@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree) tag = "DROP SUBSCRIPTION"; break; - case T_AlterCollationStmt: - tag = "ALTER COLLATION"; - break; - case T_PrepareStmt: tag = "PREPARE"; break; @@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree) lev = LOGSTMT_DDL; break; - case T_AlterCollationStmt: - lev = LOGSTMT_DDL; - break; - /* already-planned queries */ case T_PlannedStmt: { diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c index 25fb7e2ebf..597c1241f9 100644 --- a/src/backend/utils/adt/pg_locale.c +++ b/src/backend/utils/adt/pg_locale.c @@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid) const char *collctype pg_attribute_unused(); struct pg_locale_struct result; pg_locale_t resultp; - Datum collversion; - bool isnull; tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid)); if (!HeapTupleIsValid(tp)) @@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid) #endif /* not USE_ICU */ } - collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion, - &isnull); - if (!isnull) - { - char *actual_versionstr; - char *collversionstr; - - actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate); - if (!actual_versionstr) - { - /* - * This could happen when specifying a version in CREATE - * COLLATION for a libc locale, or manually creating a mess in - * the catalogs. - */ - ereport(ERROR, - (errmsg("collation \"%s\" has no actual version, but a version was specified", - NameStr(collform->collname)))); - } - collversionstr = TextDatumGetCString(collversion); - - if (strcmp(actual_versionstr, collversionstr) != 0) - ereport(WARNING, - (errmsg("collation \"%s\" has version mismatch", - NameStr(collform->collname)), - errdetail("The collation in the database was created using version %s, " - "but the operating system provides version %s.", - collversionstr, actual_versionstr), - errhint("Rebuild all objects affected by this collation and run " - "ALTER COLLATION %s REFRESH VERSION, " - "or build PostgreSQL with the right library version.", - quote_qualified_identifier(get_namespace_name(collform->collnamespace), - NameStr(collform->collname))))); - } - ReleaseSysCache(tp); /* We'll keep the pg_locale_t structures in TopMemoryContext */ diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index ec3e2c63b0..9aa6496814 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo) /* Get collation-specific details */ appendPQExpBufferStr(query, "SELECT "); - if (fout->remoteVersion >= 100000) + if (fout->remoteVersion >= 130000) + appendPQExpBufferStr(query, + "collprovider, " + "NULL AS collversion, "); + else if (fout->remoteVersion >= 100000) appendPQExpBufferStr(query, "collprovider, " "collversion, "); @@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo) * For binary upgrade, carry over the collation version. For normal * dump/restore, omit the version, so that it is computed upon restore. */ - if (dopt->binary_upgrade) + if (dopt->binary_upgrade && fout->remoteVersion < 130000) { int i_collversion; diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat index ba1b3e201b..45301ccdd7 100644 --- a/src/include/catalog/pg_collation.dat +++ b/src/include/catalog/pg_collation.dat @@ -15,17 +15,16 @@ { oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID', descr => 'database\'s default collation', collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID', - collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '', - collversion => '_null_' }, + collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' }, { oid => '950', oid_symbol => 'C_COLLATION_OID', descr => 'standard C collation', collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID', collprovider => 'c', collencoding => '-1', collcollate => 'C', - collctype => 'C', collversion => '_null_' }, + collctype => 'C' }, { oid => '951', oid_symbol => 'POSIX_COLLATION_OID', descr => 'standard POSIX collation', collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID', collprovider => 'c', collencoding => '-1', collcollate => 'POSIX', - collctype => 'POSIX', collversion => '_null_' }, + collctype => 'POSIX' }, ] diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h index 6955bb1273..cfde555366 100644 --- a/src/include/catalog/pg_collation.h +++ b/src/include/catalog/pg_collation.h @@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId) int32 collencoding; /* encoding for this collation; -1 = "all" */ NameData collcollate; /* LC_COLLATE setting */ NameData collctype; /* LC_CTYPE setting */ -#ifdef CATALOG_VARLEN /* variable-length fields start here */ - text collversion; /* provider-dependent version of collation - * data */ -#endif } FormData_pg_collation; /* ---------------- @@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace, bool collisdeterministic, int32 collencoding, const char *collcollate, const char *collctype, - const char *collversion, bool if_not_exists, bool quiet); extern void RemoveCollationById(Oid collationOid); diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h index 51491c4513..8f131893dc 100644 --- a/src/include/catalog/toasting.h +++ b/src/include/catalog/toasting.h @@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName, /* normal catalogs */ DECLARE_TOAST(pg_aggregate, 4159, 4160); DECLARE_TOAST(pg_attrdef, 2830, 2831); -DECLARE_TOAST(pg_collation, 4161, 4162); DECLARE_TOAST(pg_constraint, 2832, 2833); DECLARE_TOAST(pg_default_acl, 4143, 4144); DECLARE_TOAST(pg_description, 2834, 2835); diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h index df7d1d498c..9a9e145b4c 100644 --- a/src/include/commands/collationcmds.h +++ b/src/include/commands/collationcmds.h @@ -20,6 +20,5 @@ extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists); extern void IsThereCollationInNamespace(const char *collname, Oid nspOid); -extern ObjectAddress AlterCollation(AlterCollationStmt *stmt); #endif /* COLLATIONCMDS_H */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index da0706add5..c96d027362 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ } AlterTableCmd; -/* ---------------------- - * Alter Collation - * ---------------------- - */ -typedef struct AlterCollationStmt -{ - NodeTag type; - List *collname; -} AlterCollationStmt; - - /* ---------------------- * Alter Domain * diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out index 2b86ce9028..60d9263a2f 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8.out @@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%'; DROP SCHEMA test_schema; DROP ROLE regress_test_role; --- ALTER -ALTER COLLATION "en-x-icu" REFRESH VERSION; -NOTICE: version has not changed -- dependencies CREATE COLLATION test0 FROM "C"; CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0); diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql index 67de7d9794..35acf91fbf 100644 --- a/src/test/regress/sql/collate.icu.utf8.sql +++ b/src/test/regress/sql/collate.icu.utf8.sql @@ -405,11 +405,6 @@ DROP SCHEMA test_schema; DROP ROLE regress_test_role; --- ALTER - -ALTER COLLATION "en-x-icu" REFRESH VERSION; - - -- dependencies CREATE COLLATION test0 FROM "C"; -- 2.20.1 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0002-Add-pg_depend.refobjversion-v8.patch" From 0920cb9c88db698efd4e0b4c4b73cc800ccc9f17 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 28 May 2019 14:16:03 -0400 Subject: [PATCH 2/6] Add pg_depend.refobjversion. Provide a place for the version of referenced database objects to be recorded. Later commits will be able to use this to record dependencies on collation versions, for indexes, check constraints and so forth. Author: Thomas Munro Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- src/backend/catalog/dependency.c | 11 +++--- src/backend/catalog/pg_depend.c | 28 ++++++++++++--- src/bin/initdb/initdb.c | 44 +++++++++++------------ src/include/catalog/dependency.h | 6 ++++ src/include/catalog/pg_depend.h | 3 ++ src/test/regress/expected/misc_sanity.out | 7 ++-- 6 files changed, 66 insertions(+), 33 deletions(-) diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c index c4a4df25b8..78c31baa34 100644 --- a/src/backend/catalog/dependency.c +++ b/src/backend/catalog/dependency.c @@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender, /* And record 'em */ recordMultipleDependencies(depender, - context.addrs->refs, context.addrs->numrefs, + context.addrs->refs, NULL, + context.addrs->numrefs, behavior); free_object_addresses(context.addrs); @@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender, /* Record the self-dependencies with the appropriate direction */ if (!reverse_self) recordMultipleDependencies(depender, - self_addrs->refs, self_addrs->numrefs, + self_addrs->refs, NULL, + self_addrs->numrefs, self_behavior); else { @@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender, /* Record the external dependencies */ recordMultipleDependencies(depender, - context.addrs->refs, context.addrs->numrefs, + context.addrs->refs, NULL, + context.addrs->numrefs, behavior); free_object_addresses(context.addrs); @@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender, { eliminate_duplicate_dependencies(referenced); recordMultipleDependencies(depender, - referenced->refs, referenced->numrefs, + referenced->refs, NULL, referenced->numrefs, behavior); } diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c index f9af245eec..7fdbdf0ae8 100644 --- a/src/backend/catalog/pg_depend.c +++ b/src/backend/catalog/pg_depend.c @@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender, const ObjectAddress *referenced, DependencyType behavior) { - recordMultipleDependencies(depender, referenced, 1, behavior); + recordMultipleDependencies(depender, referenced, NULL, 1, behavior); +} + +/* + * As recordDependencyOn(), but also capture a version string so that changes + * in the referenced object can be detected. The meaning of the version + * string depends on the referenced object. Currently it is used for + * detecting changes in collation versions. + */ +void +recordDependencyOnVersion(const ObjectAddress *depender, + const ObjectAddress *referenced, + const NameData *version, + DependencyType behavior) +{ + recordMultipleDependencies(depender, referenced, version, 1, behavior); } /* @@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender, void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, + const NameData *version, int nreferenced, DependencyType behavior) { @@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender, /* Don't open indexes unless we need to make an update */ indstate = NULL; - memset(nulls, false, sizeof(nulls)); - for (i = 0; i < nreferenced; i++, referenced++) { /* @@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender, * Record the Dependency. Note we don't bother to check for * duplicate dependencies; there's no harm in them. */ + memset(nulls, false, sizeof(nulls)); + memset(values, 0, sizeof(values)); + values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId); values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId); values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId); @@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender, values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId); values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId); values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId); + if (version) + values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version); + else + nulls[Anum_pg_depend_refobjversion - 1] = true; values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior); - tup = heap_form_tuple(dependDesc->rd_att, values, nulls); /* fetch index info only when we know we need it */ diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c index 7f1534aebb..e72f85d8f0 100644 --- a/src/bin/initdb/initdb.c +++ b/src/bin/initdb/initdb.c @@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd) "DELETE FROM pg_shdepend;\n\n", "VACUUM pg_shdepend;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_class;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_proc;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_type;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_cast;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_constraint;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_conversion;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_attrdef;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_language;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_operator;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_opclass;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_opfamily;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_am;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_amop;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_amproc;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_rewrite;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_trigger;\n\n", /* * restriction here to avoid pinning the public namespace */ - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_namespace " " WHERE nspname LIKE 'pg%';\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_ts_parser;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_ts_dict;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_ts_template;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_ts_config;\n\n", - "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' " + "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL" " FROM pg_collation;\n\n", "INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' " " FROM pg_authid;\n\n", diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h index 0cd6fcf027..77cf0612ed 100644 --- a/src/include/catalog/dependency.h +++ b/src/include/catalog/dependency.h @@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender, const ObjectAddress *referenced, DependencyType behavior); +extern void recordDependencyOnVersion(const ObjectAddress *depender, + const ObjectAddress *referenced, + const NameData *version, + DependencyType behavior); + extern void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, + const NameData *version, int nreferenced, DependencyType behavior); diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h index ccf0a98330..9f2e10d428 100644 --- a/src/include/catalog/pg_depend.h +++ b/src/include/catalog/pg_depend.h @@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId) * field. See DependencyType in catalog/dependency.h. */ char deptype; /* see codes in dependency.h */ +#ifdef CATALOG_VARLEN + text refobjversion; /* version tracking, NULL if not used or unknown */ +#endif } FormData_pg_depend; /* ---------------- diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out index 8538173ff8..41efb4a2c8 100644 --- a/src/test/regress/expected/misc_sanity.out +++ b/src/test/regress/expected/misc_sanity.out @@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR (deptype != 'p' AND (classid = 0 OR objid = 0)) OR (deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0)); - classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype ----------+-------+----------+------------+----------+-------------+--------- + classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion +---------+-------+----------+------------+----------+-------------+---------+--------------- (0 rows) -- **************** pg_shdepend **************** @@ -103,9 +103,10 @@ ORDER BY 1, 2; pg_class | relacl | aclitem[] pg_class | reloptions | text[] pg_class | relpartbound | pg_node_tree + pg_depend | refobjversion | text pg_index | indexprs | pg_node_tree pg_index | indpred | pg_node_tree pg_largeobject | data | bytea pg_largeobject_metadata | lomacl | aclitem[] -(11 rows) +(12 rows) -- 2.20.1 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0003-Implement-type-regcollation-v8.patch" From 988b4a7c43fbeaf644859b31582cf47fd4d992e3 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Thu, 5 Dec 2019 18:59:28 +0100 Subject: [PATCH 3/6] Implement type regcollation. This will be helpful for a following commit. Author: Julien Rouhaud Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++ src/include/catalog/pg_cast.dat | 14 +++ src/include/catalog/pg_proc.dat | 15 ++++ src/include/catalog/pg_type.dat | 4 + 4 files changed, 185 insertions(+) diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c index f0fa52bc27..da8cc0cf6b 100644 --- a/src/backend/utils/adt/regproc.c +++ b/src/backend/utils/adt/regproc.c @@ -24,6 +24,7 @@ #include "access/htup_details.h" #include "catalog/namespace.h" #include "catalog/pg_class.h" +#include "catalog/pg_collation.h" #include "catalog/pg_operator.h" #include "catalog/pg_proc.h" #include "catalog/pg_ts_config.h" @@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS) } +/* + * regcollationin - converts "collationname" to collation OID + * + * We also accept a numeric OID, for symmetry with the output routine. + * + * '-' signifies unknown (OID 0). In all other cases, the input must + * match an existing pg_collation entry. + */ +Datum +regcollationin(PG_FUNCTION_ARGS) +{ + char *collation_name_or_oid = PG_GETARG_CSTRING(0); + Oid result = InvalidOid; + List *names; + + /* '-' ? */ + if (strcmp(collation_name_or_oid, "-") == 0) + PG_RETURN_OID(InvalidOid); + + /* Numeric OID? */ + if (collation_name_or_oid[0] >= '0' && + collation_name_or_oid[0] <= '9' && + strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid)) + { + result = DatumGetObjectId(DirectFunctionCall1(oidin, + CStringGetDatum(collation_name_or_oid))); + PG_RETURN_OID(result); + } + + /* Else it's a name, possibly schema-qualified */ + + /* The rest of this wouldn't work in bootstrap mode */ + if (IsBootstrapProcessingMode()) + elog(ERROR, "regcollation values must be OIDs in bootstrap mode"); + + /* + * Normal case: parse the name into components and see if it matches any + * pg_collation entries in the current search path. + */ + names = stringToQualifiedNameList(collation_name_or_oid); + + result = get_collation_oid(names, false); + + PG_RETURN_OID(result); +} + +/* + * to_regcollation - converts "collationname" to collation OID + * + * If the name is not found, we return NULL. + */ +Datum +to_regcollation(PG_FUNCTION_ARGS) +{ + char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0)); + Oid result; + List *names; + + /* + * Parse the name into components and see if it matches any pg_collation + * entries in the current search path. + */ + names = stringToQualifiedNameList(collation_name); + + /* We might not even have permissions on this relation; don't lock it. */ + result = get_collation_oid(names, true); + + if (OidIsValid(result)) + PG_RETURN_OID(result); + else + PG_RETURN_NULL(); +} + +/* + * regcollationout - converts collation OID to "collation_name" + */ +Datum +regcollationout(PG_FUNCTION_ARGS) +{ + Oid collationid = PG_GETARG_OID(0); + char *result; + HeapTuple collationtup; + + if (collationid == InvalidOid) + { + result = pstrdup("-"); + PG_RETURN_CSTRING(result); + } + + collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid)); + + if (HeapTupleIsValid(collationtup)) + { + Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup); + char *collationname = NameStr(collationform->collname); + + /* + * In bootstrap mode, skip the fancy namespace stuff and just return + * the collation name. (This path is only needed for debugging output + * anyway.) + */ + if (IsBootstrapProcessingMode()) + result = pstrdup(collationname); + else + { + char *nspname; + + /* + * Would this collation be found by regcollationin? If not, qualify it. + */ + if (CollationIsVisible(collationid)) + nspname = NULL; + else + nspname = get_namespace_name(collationform->collnamespace); + + result = quote_qualified_identifier(nspname, collationname); + } + + ReleaseSysCache(collationtup); + } + else + { + /* If OID doesn't match any pg_collation entry, return it numerically */ + result = (char *) palloc(NAMEDATALEN); + snprintf(result, NAMEDATALEN, "%u", collationid); + } + + PG_RETURN_CSTRING(result); +} + +/* + * regcollationrecv - converts external binary format to regcollation + */ +Datum +regcollationrecv(PG_FUNCTION_ARGS) +{ + /* Exactly the same as oidrecv, so share code */ + return oidrecv(fcinfo); +} + +/* + * regcollationsend - converts regcollation to binary format + */ +Datum +regcollationsend(PG_FUNCTION_ARGS) +{ + /* Exactly the same as oidsend, so share code */ + return oidsend(fcinfo); +} + + /* * regtypein - converts "typename" to type OID * diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat index 6ef8b8a4e7..01c5328ddd 100644 --- a/src/include/catalog/pg_cast.dat +++ b/src/include/catalog/pg_cast.dat @@ -189,6 +189,20 @@ castcontext => 'a', castmethod => 'f' }, { castsource => 'regclass', casttarget => 'int4', castfunc => '0', castcontext => 'a', castmethod => 'b' }, +{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0', + castcontext => 'i', castmethod => 'b' }, +{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0', + castcontext => 'i', castmethod => 'b' }, +{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid', + castcontext => 'i', castmethod => 'f' }, +{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)', + castcontext => 'i', castmethod => 'f' }, +{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0', + castcontext => 'i', castmethod => 'b' }, +{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)', + castcontext => 'a', castmethod => 'f' }, +{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0', + castcontext => 'a', castmethod => 'b' }, { castsource => 'oid', casttarget => 'regtype', castfunc => '0', castcontext => 'i', castmethod => 'b' }, { castsource => 'regtype', casttarget => 'oid', castfunc => '0', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 226c904c04..f6503428cc 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6663,6 +6663,15 @@ { oid => '3495', descr => 'convert classname to regclass', proname => 'to_regclass', provolatile => 's', prorettype => 'regclass', proargtypes => 'text', prosrc => 'to_regclass' }, +{ oid => '9508', descr => 'I/O', + proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation', + proargtypes => 'cstring', prosrc => 'regcollationin' }, +{ oid => '9509', descr => 'I/O', + proname => 'regcollationout', provolatile => 's', prorettype => 'cstring', + proargtypes => 'regcollation', prosrc => 'regcollationout' }, +{ oid => '9510', descr => 'convert classname to regcollation', + proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation', + proargtypes => 'text', prosrc => 'to_regcollation' }, { oid => '2220', descr => 'I/O', proname => 'regtypein', provolatile => 's', prorettype => 'regtype', proargtypes => 'cstring', prosrc => 'regtypein' }, @@ -7449,6 +7458,12 @@ { oid => '2453', descr => 'I/O', proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass', prosrc => 'regclasssend' }, +{ oid => '9511', descr => 'I/O', + proname => 'regcollationrecv', prorettype => 'regcollation', + proargtypes => 'internal', prosrc => 'regcollationrecv' }, +{ oid => '9512', descr => 'I/O', + proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation', + prosrc => 'regcollationsend' }, { oid => '2454', descr => 'I/O', proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal', prosrc => 'regtyperecv' }, diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index fe2c4eabb4..012c17bb68 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -379,6 +379,10 @@ typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N', typinput => 'regclassin', typoutput => 'regclassout', typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' }, +{ oid => '9506', array_type_oid => '9507', descr => 'registered collation', + typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N', + typinput => 'regcollationin', typoutput => 'regcollationout', + typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' }, { oid => '2206', array_type_oid => '2211', descr => 'registered type', typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N', typinput => 'regtypein', typoutput => 'regtypeout', -- 2.20.1 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0004-Track-collation-versions-for-indexes-v8.patch" From cf3b46a0727366e00683421f45fbd6a228106b53 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 28 May 2019 14:16:29 -0400 Subject: [PATCH 4/6] Track collation versions for indexes. Record the current version of dependent collations in pg_depend when creating or rebuilding an index. That version is checked against current version whenever we call get_relation_info for an index or open the parent table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if the versions don't match. A new flag is added in RelationData to specify that the check has already beed done to avoid checking and reporting the message multiple time in a backend lifetime. Author: Thomas Munro, Julien Rouhaud Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- src/backend/catalog/dependency.c | 189 ++++++++++++++++-- src/backend/catalog/heap.c | 7 +- src/backend/catalog/index.c | 139 ++++++++++++- src/backend/catalog/pg_constraint.c | 2 +- src/backend/catalog/pg_depend.c | 127 ++++++++++-- src/backend/catalog/pg_type.c | 69 +++++++ src/backend/commands/vacuum.c | 31 +++ src/backend/optimizer/util/plancat.c | 9 + src/backend/utils/adt/pg_locale.c | 38 ++++ src/backend/utils/cache/relcache.c | 2 + src/include/catalog/dependency.h | 21 +- src/include/catalog/index.h | 2 + src/include/catalog/pg_type.h | 2 + src/include/utils/pg_locale.h | 1 + src/include/utils/rel.h | 1 + .../regress/expected/collate.icu.utf8.out | 118 +++++++++++ ...te.icu.utf8.out => collate.icu.utf8_2.out} | 118 +++++++++++ src/test/regress/expected/create_index.out | 8 +- src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++ 19 files changed, 901 insertions(+), 59 deletions(-) copy src/test/regress/expected/{collate.icu.utf8.out => collate.icu.utf8_2.out} (89%) diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c index 78c31baa34..93f57cd633 100644 --- a/src/backend/catalog/dependency.c +++ b/src/backend/catalog/dependency.c @@ -77,6 +77,7 @@ #include "parser/parsetree.h" #include "rewrite/rewriteRemove.h" #include "storage/lmgr.h" +#include "utils/builtins.h" #include "utils/fmgroids.h" #include "utils/guc.h" #include "utils/lsyscache.h" @@ -137,6 +138,9 @@ typedef struct { ObjectAddresses *addrs; /* addresses being accumulated */ List *rtables; /* list of rangetables to resolve Vars */ + bool track_version; /* whether caller asked to track dependency + * versions */ + NodeTag type; /* nodetag of the current node */ } find_expr_references_context; /* @@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects, table_close(depRel, RowExclusiveLock); } +/* + * Call a function for all objects that depend on 'object'. If the function + * returns a non-NULL pointer to a new version string, update the version. + */ +void +visitDependentObjects(const ObjectAddress *object, + VisitDependentObjectsFun callback, + void *userdata) +{ + Relation depRel; + ScanKeyData key[3]; + SysScanDesc scan; + HeapTuple tup; + ObjectAddress otherObject; + + ScanKeyInit(&key[0], + Anum_pg_depend_classid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(object->classId)); + ScanKeyInit(&key[1], + Anum_pg_depend_objid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(object->objectId)); + ScanKeyInit(&key[2], + Anum_pg_depend_objsubid, + BTEqualStrategyNumber, F_INT4EQ, + Int32GetDatum(object->objectSubId)); + + depRel = table_open(DependRelationId, RowExclusiveLock); + scan = systable_beginscan(depRel, DependDependerIndexId, true, + NULL, 3, key); + + while (HeapTupleIsValid(tup = systable_getnext(scan))) + { + Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup); + char *cur_version, *new_version; + Datum depversion; + bool isnull; + + otherObject.classId = foundDep->refclassid; + otherObject.objectId = foundDep->refobjid; + otherObject.objectSubId = foundDep->refobjsubid; + + depversion = heap_getattr(tup, Anum_pg_depend_refobjversion, + RelationGetDescr(depRel), &isnull); + + cur_version = isnull ? NULL : TextDatumGetCString(depversion); + + new_version = callback(&otherObject, cur_version, userdata); + if (new_version) + { + Datum values[Natts_pg_depend]; + bool nulls[Natts_pg_depend]; + bool replaces[Natts_pg_depend]; + + memset(values, 0, sizeof(values)); + memset(nulls, false, sizeof(nulls)); + memset(replaces, false, sizeof(replaces)); + + values[Anum_pg_depend_refobjversion - 1] = + CStringGetTextDatum(new_version); + replaces[Anum_pg_depend_refobjversion - 1] = true; + + tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values, + nulls, replaces); + CatalogTupleUpdate(depRel, &tup->t_self, tup); + + heap_freetuple(tup); + } + } + systable_endscan(scan); + table_close(depRel, RowExclusiveLock); +} + /* * findDependentObjects - find all objects that depend on 'object' * @@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender, find_expr_references_context context; context.addrs = new_object_addresses(); + if (expr) + context.type = expr->type; + else + context.type = T_Invalid; /* Set up interpretation for Vars at varlevelsup = 0 */ context.rtables = list_make1(rtable); @@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender, /* And record 'em */ recordMultipleDependencies(depender, - context.addrs->refs, NULL, + context.addrs->refs, context.addrs->numrefs, - behavior); + behavior, + false); free_object_addresses(context.addrs); } @@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender, Node *expr, Oid relId, DependencyType behavior, DependencyType self_behavior, - bool reverse_self) + bool reverse_self, + bool track_version) { find_expr_references_context context; RangeTblEntry rte; context.addrs = new_object_addresses(); + context.track_version = track_version; + if (expr) + context.type = expr->type; + else + context.type = T_Invalid; /* We gin up a rather bogus rangetable list to handle Vars */ MemSet(&rte, 0, sizeof(rte)); @@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender, /* Record the self-dependencies with the appropriate direction */ if (!reverse_self) recordMultipleDependencies(depender, - self_addrs->refs, NULL, + self_addrs->refs, self_addrs->numrefs, - self_behavior); + self_behavior, + track_version); else { /* Can't use recordMultipleDependencies, so do it the hard way */ @@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender, /* Record the external dependencies */ recordMultipleDependencies(depender, - context.addrs->refs, NULL, + context.addrs->refs, context.addrs->numrefs, - behavior); + behavior, + track_version); free_object_addresses(context.addrs); } @@ -1735,8 +1826,13 @@ static bool find_expr_references_walker(Node *node, find_expr_references_context *context) { + NodeTag parent_type = context->type; + if (node == NULL) return false; + + context->type = node->type; + if (IsA(node, Var)) { Var *var = (Var *) node; @@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node, /* If it's a plain relation, reference this column */ add_object_address(OCLASS_CLASS, rte->relid, var->varattno, context->addrs); + + /* Record collations from the type itself, or underlying in case of + * complex type. Note that if the direct parent is a CollateExpr + * node, there's no need to record the type underlying collation if + * any. A dependency already exists for the owning relation, and a + * change in the collation sort order wouldn't cause any harm as + * the collation isn't used at all in such case. + */ + if (parent_type != T_CollateExpr) + { + /* type's collation if valid */ + if (OidIsValid(var->varcollid)) + { + add_object_address(OCLASS_COLLATION, var->varcollid, 0, + context->addrs); + } + /* + * otherwise, it may be a composite type having underlying + * collations */ + else if (var->vartype >= FirstNormalObjectId) + { + List *collations = NIL; + ListCell *lc; + + collations = GetTypeCollations(var->vartype, false); + + foreach(lc, collations) + { + Oid coll = lfirst_oid(lc); + + if (OidIsValid(coll) && + (coll != DEFAULT_COLLATION_OID || + context->track_version) + ) + add_object_address(OCLASS_COLLATION, + lfirst_oid(lc), 0, + context->addrs); + } + } + } } /* @@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node, /* * We must also depend on the constant's collation: it could be * different from the datatype's, if a CollateExpr was const-folded to - * a simple constant. However we can save work in the most common - * case where the collation is "default", since we know that's pinned. + * a simple constant. However we can save work in the most common case + * where the collation is "default", since we know that's pinned, if + * the caller didn't ask to track dependency versions. */ if (OidIsValid(con->constcollid) && - con->constcollid != DEFAULT_COLLATION_OID) + (con->constcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, con->constcollid, 0, context->addrs); @@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node, context->addrs); /* and its collation, just as for Consts */ if (OidIsValid(param->paramcollid) && - param->paramcollid != DEFAULT_COLLATION_OID) + (param->paramcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, param->paramcollid, 0, context->addrs); } @@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node, context->addrs); /* the collation might not be referenced anywhere else, either */ if (OidIsValid(fselect->resultcollid) && - fselect->resultcollid != DEFAULT_COLLATION_OID) + (fselect->resultcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0, context->addrs); } @@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node, context->addrs); /* the collation might not be referenced anywhere else, either */ if (OidIsValid(relab->resultcollid) && - relab->resultcollid != DEFAULT_COLLATION_OID) + (relab->resultcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, relab->resultcollid, 0, context->addrs); } @@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node, context->addrs); /* the collation might not be referenced anywhere else, either */ if (OidIsValid(iocoerce->resultcollid) && - iocoerce->resultcollid != DEFAULT_COLLATION_OID) + (iocoerce->resultcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0, context->addrs); } @@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node, context->addrs); /* the collation might not be referenced anywhere else, either */ if (OidIsValid(acoerce->resultcollid) && - acoerce->resultcollid != DEFAULT_COLLATION_OID) + (acoerce->resultcollid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0, context->addrs); /* fall through to examine arguments */ @@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node, add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0, context->addrs); if (OidIsValid(wc->inRangeColl) && - wc->inRangeColl != DEFAULT_COLLATION_OID) + (wc->inRangeColl != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0, context->addrs); /* fall through to examine substructure */ @@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node, { Oid collid = lfirst_oid(ct); - if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID) + if (OidIsValid(collid) && + (collid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, collid, 0, context->addrs); } @@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node, { Oid collid = lfirst_oid(ct); - if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID) + if (OidIsValid(collid) && + (collid != DEFAULT_COLLATION_OID || + context->track_version)) add_object_address(OCLASS_COLLATION, collid, 0, context->addrs); } @@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender, { eliminate_duplicate_dependencies(referenced); recordMultipleDependencies(depender, - referenced->refs, NULL, referenced->numrefs, - behavior); + referenced->refs, referenced->numrefs, + behavior, + false); } /* diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index e31478bf91..46e9d74a97 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, */ recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel), DEPENDENCY_AUTO, - DEPENDENCY_AUTO, false); + DEPENDENCY_AUTO, false, false); } else { @@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, */ recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel), DEPENDENCY_NORMAL, - DEPENDENCY_NORMAL, false); + DEPENDENCY_NORMAL, false, false); } /* @@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel, RelationGetRelid(rel), DEPENDENCY_NORMAL, DEPENDENCY_INTERNAL, - true /* reverse the self-deps */ ); + true /* reverse the self-deps */, + false /* don't track versions */); /* * We must invalidate the relcache so that the next diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 8880586c37..cd5c93d4ef 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -74,6 +74,7 @@ #include "utils/guc.h" #include "utils/inval.h" #include "utils/lsyscache.h" +#include "utils/pg_locale.h" #include "utils/memutils.h" #include "utils/pg_rusage.h" #include "utils/snapmgr.h" @@ -724,6 +725,7 @@ index_create(Relation heapRelation, char relkind; TransactionId relfrozenxid; MultiXactId relminmxid; + List *collations = NIL; /* constraint flags can only be set when a constraint is requested */ Assert((constr_flags == 0) || @@ -1115,21 +1117,44 @@ index_create(Relation heapRelation, recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC); } - /* Store dependency on collations */ - /* The default collation is pinned, so don't bother recording it */ + /* + * Get required distinct dependencies on collations for all index keys. + * Collations of directly referenced column in hash indexes can be + * skipped is they're deterministic. + */ for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++) { - if (OidIsValid(collationObjectId[i]) && - collationObjectId[i] != DEFAULT_COLLATION_OID) + Oid colloid = collationObjectId[i]; + + if (OidIsValid(colloid)) { - referenced.classId = CollationRelationId; - referenced.objectId = collationObjectId[i]; - referenced.objectSubId = 0; + if ((indexInfo->ii_Am != HASH_AM_OID) || + !get_collation_isdeterministic(colloid)) + collations = list_append_unique_oid(collations, colloid); + } + else + { + Form_pg_attribute att = TupleDescAttr(indexTupDesc, i); + + Assert(i < indexTupDesc->natts); - recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + collations = list_concat_unique_oid(collations, + GetTypeCollations(att->atttypid, + (indexInfo->ii_Am == HASH_AM_OID))); } } + if (collations) + { + recordDependencyOnCollations(&myself, collations); + /* + * Advance the command counter so that later calls to + * recordMultipleDependencies calls can see the newly-entered + * pg_depend catalog tuples for the index. + */ + CommandCounterIncrement(); + } + /* Store dependency on operator classes */ for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++) { @@ -1143,21 +1168,29 @@ index_create(Relation heapRelation, /* Store dependencies on anything mentioned in index expressions */ if (indexInfo->ii_Expressions) { + /* recordDependencyOnSingleRelExpr get rid of duplicate entries */ recordDependencyOnSingleRelExpr(&myself, (Node *) indexInfo->ii_Expressions, heapRelationId, DEPENDENCY_NORMAL, - DEPENDENCY_AUTO, false); + DEPENDENCY_AUTO, false, true); + /* + * Advance the command counter so that later + * recordMultipleDependencies calls can see the newly-entered + * pg_depend catalog tuples for the index. + */ + CommandCounterIncrement(); } /* Store dependencies on anything mentioned in predicate */ if (indexInfo->ii_Predicate) { + /* recordDependencyOnSingleRelExpr get rid of duplicate entries */ recordDependencyOnSingleRelExpr(&myself, (Node *) indexInfo->ii_Predicate, heapRelationId, DEPENDENCY_NORMAL, - DEPENDENCY_AUTO, false); + DEPENDENCY_AUTO, false, true); } } else @@ -1229,6 +1262,89 @@ index_create(Relation heapRelation, return indexRelationId; } +static char * +index_check_collation_version(const ObjectAddress *otherObject, + const char *version, + void *userdata) +{ + Oid relid = *(Oid *) userdata; + char *current_version; + + /* We only care about dependencies on collations. */ + if (otherObject->classId != CollationRelationId) + return NULL; + + /* Compare with the current version. */ + current_version = get_collation_version_for_oid(otherObject->objectId); + + /* XXX should we warn about "disappearing" versions? */ + if (current_version) + { + if (!version) + { + ereport(WARNING, + (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"", + get_rel_name(relid), + get_collation_name(otherObject->objectId), + current_version), + errdetail("The index may be corrupted due to changes in sort order."), + errhint("REINDEX to avoid the risk of corruption."))); + } + else if (strcmp(current_version, version) != 0) + { + ereport(WARNING, + (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"", + get_rel_name(relid), + get_collation_name(otherObject->objectId), + version, + current_version), + errdetail("The index may be corrupted due to changes in sort order."), + errhint("REINDEX to avoid the risk of corruption."))); + } + } + + return NULL; +} + +void +index_check_collation_versions(Oid relid) +{ + ObjectAddress object; + + object.classId = RelationRelationId; + object.objectId = relid; + object.objectSubId = 0; + visitDependentObjects(&object, &index_check_collation_version, &relid); +} + +static char * +index_update_collation_version(const ObjectAddress *otherObject, + const char *version, + void *userdata) +{ + char *current_version = (char *) userdata; + + /* We only care about dependencies on collations. */ + if (otherObject->classId != CollationRelationId) + return NULL; + + current_version = get_collation_version_for_oid(otherObject->objectId); + return current_version; +} + +static void +index_update_collation_versions(Oid relid) +{ + ObjectAddress object; + NameData current_version; + + object.classId = RelationRelationId; + object.objectId = relid; + object.objectSubId = 0; + visitDependentObjects(&object, &index_update_collation_version, + ¤t_version); +} + /* * index_concurrently_create_copy * @@ -3605,6 +3721,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence, /* Close rels, but keep locks */ index_close(iRel, NoLock); table_close(heapRelation, NoLock); + + /* Record the current versions of all depended-on collations. */ + index_update_collation_versions(indexId); } /* diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c index 3d2b1cc911..0eaefbd032 100644 --- a/src/backend/catalog/pg_constraint.c +++ b/src/backend/catalog/pg_constraint.c @@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName, */ recordDependencyOnSingleRelExpr(&conobject, conExpr, relId, DEPENDENCY_NORMAL, - DEPENDENCY_NORMAL, false); + DEPENDENCY_NORMAL, false, true); } /* Post creation hook for new constraint */ diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c index 7fdbdf0ae8..709e7f5530 100644 --- a/src/backend/catalog/pg_depend.c +++ b/src/backend/catalog/pg_depend.c @@ -19,16 +19,21 @@ #include "access/table.h" #include "catalog/dependency.h" #include "catalog/indexing.h" +#include "catalog/pg_collation.h" #include "catalog/pg_constraint.h" #include "catalog/pg_depend.h" #include "catalog/pg_extension.h" #include "commands/extension.h" #include "miscadmin.h" +#include "utils/builtins.h" #include "utils/fmgroids.h" #include "utils/lsyscache.h" +#include "utils/pg_locale.h" #include "utils/rel.h" +static bool dependencyExists(const ObjectAddress *depender, + const ObjectAddress *referenced); static bool isObjectPinned(const ObjectAddress *object, Relation rel); @@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender, const ObjectAddress *referenced, DependencyType behavior) { - recordMultipleDependencies(depender, referenced, NULL, 1, behavior); + recordMultipleDependencies(depender, referenced, 1, behavior, false); } /* - * As recordDependencyOn(), but also capture a version string so that changes - * in the referenced object can be detected. The meaning of the version - * string depends on the referenced object. Currently it is used for - * detecting changes in collation versions. + * Given a list of collations, record a dependency on its underlying collation + * version. */ -void -recordDependencyOnVersion(const ObjectAddress *depender, - const ObjectAddress *referenced, - const NameData *version, - DependencyType behavior) +void recordDependencyOnCollations(ObjectAddress *myself, + List *collations) { - recordMultipleDependencies(depender, referenced, version, 1, behavior); + ListCell *lc; + + foreach(lc, collations) + { + ObjectAddress referenced; + + referenced.classId = CollationRelationId; + referenced.objectId = lfirst_oid(lc); + referenced.objectSubId = 0; + + recordMultipleDependencies(myself, &referenced, 1, + DEPENDENCY_NORMAL, true); + } } /* @@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender, void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, - const NameData *version, int nreferenced, - DependencyType behavior) + DependencyType behavior, + bool track_version) { Relation dependDesc; CatalogIndexState indstate; @@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender, int i; bool nulls[Natts_pg_depend]; Datum values[Natts_pg_depend]; + char *version = NULL; if (nreferenced <= 0) return; /* nothing to do */ @@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender, for (i = 0; i < nreferenced; i++, referenced++) { + bool ignore_systempin = false; + + if (track_version) + { + /* Only dependency on a collation needs to be tracked */ + if (referenced->classId == CollationRelationId) + { + /* C and POSIX collations don't require tracking the version */ + if (referenced->objectId == C_COLLATION_OID || + referenced->objectId == POSIX_COLLATION_OID) + continue; + + /* + * We don't want to record redundant depedencies that are used + * to track versions to avoid redundant warnings in case of + * non-matching versions when those are checked. Note that + * callers have to take care of removing duplicated entries and + * calling CommandCounterIncrement() if the dependencies are + * registered in multiple calls. + */ + if (dependencyExists(depender, referenced)) + continue; + + /* + * Default collation is pinned, so we need to force recording + * the dependency to store the version + */ + if (referenced->objectId == DEFAULT_COLLATION_OID) + ignore_systempin = true;; + version = get_collation_version_for_oid(referenced->objectId); + } + } + /* * If the referenced object is pinned by the system, there's no real - * need to record dependencies on it. This saves lots of space in - * pg_depend, so it's worth the time taken to check. + * need to record dependencies on it, unless we need to record a + * version. This saves lots of space in pg_depend, so it's worth the + * time taken to check. */ - if (!isObjectPinned(referenced, dependDesc)) + if (ignore_systempin || !isObjectPinned(referenced, dependDesc)) { /* * Record the Dependency. Note we don't bother to check for @@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId, return count; } +/* dependencyExists() + * + * Test if a record exists for the given depender and referenceds addresses. + */ +static bool dependencyExists(const ObjectAddress *depender, + const ObjectAddress *referenced) +{ + Relation depRel; + ScanKeyData key[3]; + SysScanDesc scan; + HeapTuple tup; + bool ret = false; + + ScanKeyInit(&key[0], + Anum_pg_depend_classid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(depender->classId)); + ScanKeyInit(&key[1], + Anum_pg_depend_objid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(depender->objectId)); + ScanKeyInit(&key[2], + Anum_pg_depend_objsubid, + BTEqualStrategyNumber, F_INT4EQ, + Int32GetDatum(depender->objectSubId)); + + depRel = table_open(DependRelationId, RowExclusiveLock); + scan = systable_beginscan(depRel, DependDependerIndexId, true, + NULL, 3, key); + + while (HeapTupleIsValid(tup = systable_getnext(scan))) + { + Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup); + + if (foundDep->refclassid == referenced->classId && + foundDep->refobjid == referenced->objectId && + foundDep->refobjsubid == referenced->objectSubId) + { + ret = true; + break; + } + } + systable_endscan(scan); + table_close(depRel, RowExclusiveLock); + + return ret; +} + /* * isObjectPinned() * diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c index 8d7572da51..6e91518694 100644 --- a/src/backend/catalog/pg_type.c +++ b/src/backend/catalog/pg_type.c @@ -15,6 +15,7 @@ #include "postgres.h" #include "access/htup_details.h" +#include "access/relation.h" #include "access/table.h" #include "access/xact.h" #include "catalog/binary_upgrade.h" @@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid, return address; } +/* Get a list of all distinct collations oid that the given type depends on. */ +List * +GetTypeCollations(Oid typeoid, bool non_deterministic_only) +{ + List *result = NIL; + HeapTuple tuple; + Form_pg_type typeTup; + + tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid)); + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for type %u", typeoid); + typeTup = (Form_pg_type) GETSTRUCT(tuple); + + if (OidIsValid(typeTup->typcollation)) + { + if (!non_deterministic_only || + !get_collation_isdeterministic(typeTup->typcollation)) + result = list_append_unique_oid(result, typeTup->typcollation); + } + else if (typeTup->typtype == TYPTYPE_COMPOSITE) + { + Relation rel = relation_open(typeTup->typrelid, AccessShareLock); + TupleDesc desc = RelationGetDescr(rel); + + for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++) + { + Form_pg_attribute att = TupleDescAttr(desc, i); + + if (OidIsValid(att->attcollation)) + { + if (!non_deterministic_only || + !get_collation_isdeterministic(att->attcollation)) + result = list_append_unique_oid(result, att->attcollation); + } + else + result = list_concat_unique_oid(result, + GetTypeCollations(att->atttypid, + non_deterministic_only)); + } + + relation_close(rel, NoLock); + } + else if (typeTup->typtype == TYPTYPE_DOMAIN) + { + Assert(OidIsValid(typeTup->typbasetype)); + + result = list_concat_unique_oid(result, + GetTypeCollations(typeTup->typbasetype, + non_deterministic_only)); + } + else if (typeTup->typtype == TYPTYPE_RANGE) + { + Oid rangeid = get_range_subtype(typeTup->oid); + + Assert(OidIsValid(rangeid)); + + result = list_concat_unique_oid(result, + GetTypeCollations(rangeid, non_deterministic_only)); + } + else if (OidIsValid(typeTup->typelem)) + result = list_concat_unique_oid(result, + GetTypeCollations(typeTup->typelem, non_deterministic_only)); + + ReleaseSysCache(tuple); + + return result; +} + /* * GenerateTypeDependencies: build the dependencies needed for a type * diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index d625d17bf4..2b7de111cd 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -31,6 +31,8 @@ #include "access/tableam.h" #include "access/transam.h" #include "access/xact.h" +#include "catalog/catalog.h" +#include "catalog/index.h" #include "catalog/namespace.h" #include "catalog/pg_database.h" #include "catalog/pg_inherits.h" @@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options, rel_lock = false; } + /* + * Perform version sanity checks on the relation underlying indexes if + * it's not a VACUUM FULL + */ + if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) && + onerel->rd_rel->relhasindex) + { + List *indexoidlist; + ListCell *l; + + indexoidlist = RelationGetIndexList(onerel); + foreach(l, indexoidlist) + { + Oid indexoid = lfirst_oid(l); + Relation indexRelation; + + indexRelation = index_open(indexoid, AccessShareLock); + + /* Warn if any dependent collations' versions have moved. */ + if (!indexRelation->rd_version_checked) + { + index_check_collation_versions(indexoid); + indexRelation->rd_version_checked = true; + } + + index_close(indexRelation, NoLock); + } + } + /* if relation is opened, leave */ if (onerel) return onerel; diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c index d82fc5ab8b..f2fc427fc8 100644 --- a/src/backend/optimizer/util/plancat.c +++ b/src/backend/optimizer/util/plancat.c @@ -28,6 +28,7 @@ #include "catalog/catalog.h" #include "catalog/dependency.h" #include "catalog/heap.h" +#include "catalog/index.h" #include "catalog/pg_am.h" #include "catalog/pg_proc.h" #include "catalog/pg_statistic_ext.h" @@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, indexRelation = index_open(indexoid, lmode); index = indexRelation->rd_index; + /* Warn if any dependent collations' versions have moved. */ + if (!IsSystemRelation(relation) && + !indexRelation->rd_version_checked) + { + index_check_collation_versions(indexoid); + indexRelation->rd_version_checked = true; + } + /* * Ignore invalid indexes, since they can't safely be used for * queries. Note that this is OK because the data structure we diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c index 597c1241f9..123bfa6c01 100644 --- a/src/backend/utils/adt/pg_locale.c +++ b/src/backend/utils/adt/pg_locale.c @@ -57,7 +57,9 @@ #include "access/htup_details.h" #include "catalog/pg_collation.h" #include "catalog/pg_control.h" +#include "catalog/pg_database.h" #include "mb/pg_wchar.h" +#include "miscadmin.h" #include "utils/builtins.h" #include "utils/formatting.h" #include "utils/hsearch.h" @@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate) return collversion; } +/* + * Get provider-specific collation version string given a collation OID. + */ +char * +get_collation_version_for_oid(Oid oid) +{ + HeapTuple tp; + char *version; + + if (oid == DEFAULT_COLLATION_OID) + { + Form_pg_database dbform; + + tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for database %u", MyDatabaseId); + dbform = (Form_pg_database) GETSTRUCT(tp); + version = get_collation_actual_version(COLLPROVIDER_LIBC, + NameStr(dbform->datcollate)); + } + else + { + Form_pg_collation collform; + + tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for collation %u", oid); + collform = (Form_pg_collation) GETSTRUCT(tp); + version = get_collation_actual_version(collform->collprovider, + NameStr(collform->collcollate)); + } + + ReleaseSysCache(tp); + + return version; +} #ifdef USE_ICU /* diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index ff70326474..bdf50ffe89 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -41,6 +41,7 @@ #include "access/xact.h" #include "access/xlog.h" #include "catalog/catalog.h" +#include "catalog/index.h" #include "catalog/indexing.h" #include "catalog/namespace.h" #include "catalog/pg_am.h" @@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared) rel->rd_idattr = NULL; rel->rd_pubactions = NULL; rel->rd_statvalid = false; + rel->rd_version_checked = false; rel->rd_statlist = NIL; rel->rd_fkeyvalid = false; rel->rd_fkeylist = NIL; diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h index 77cf0612ed..6de17e25e8 100644 --- a/src/include/catalog/dependency.h +++ b/src/include/catalog/dependency.h @@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender, Node *expr, Oid relId, DependencyType behavior, DependencyType self_behavior, - bool reverse_self); + bool reverse_self, + bool track_version); extern ObjectClass getObjectClass(const ObjectAddress *object); @@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs); extern void free_object_addresses(ObjectAddresses *addrs); +typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject, + const char *version, + void *userdata); + +extern void visitDependentObjects(const ObjectAddress *object, + VisitDependentObjectsFun callback, + void *userdata); + /* in pg_depend.c */ extern void recordDependencyOn(const ObjectAddress *depender, const ObjectAddress *referenced, DependencyType behavior); -extern void recordDependencyOnVersion(const ObjectAddress *depender, - const ObjectAddress *referenced, - const NameData *version, - DependencyType behavior); +extern void recordDependencyOnCollations(ObjectAddress *myself, + List *collations); extern void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, - const NameData *version, int nreferenced, - DependencyType behavior); + DependencyType behavior, + bool track_version); extern void recordDependencyOnCurrentExtension(const ObjectAddress *object, bool isReplace); diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index a2890c1314..c619d02465 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo, Datum *values, bool *isnull); +extern void index_check_collation_versions(Oid relid); + extern void index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo, diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h index e1a5ab3df3..2bf6f868a5 100644 --- a/src/include/catalog/pg_type.h +++ b/src/include/catalog/pg_type.h @@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId, bool isDependentType, bool rebuild); +extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only); + extern void RenameTypeInternal(Oid typeOid, const char *newTypeName, Oid typeNamespace); diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h index 9cb7d91ddf..aa06aca90a 100644 --- a/src/include/utils/pg_locale.h +++ b/src/include/utils/pg_locale.h @@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t; extern pg_locale_t pg_newlocale_from_collation(Oid collid); extern char *get_collation_actual_version(char collprovider, const char *collcollate); +extern char *get_collation_version_for_oid(Oid collid); #ifdef USE_ICU extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes); diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 44ed04dd3f..3656ea94e8 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -63,6 +63,7 @@ typedef struct RelationData bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and * rd_replidindex) */ bool rd_statvalid; /* is rd_statlist valid? */ + bool rd_version_checked; /* has version check being done yet? */ /* * rd_createSubid is the ID of the highest subtransaction the rel has diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out index 60d9263a2f..f049189897 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8.out @@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1); t (1 row) +-- collation versioning support +CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu"); +CREATE DOMAIN d_en_fr AS t_en_fr; +CREATE DOMAIN d_es AS text COLLATE "es-x-icu"; +CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu"); +CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga; +CREATE TYPE t_custom AS (meh text, meh2 text); +CREATE DOMAIN d_custom AS t_custom; +CREATE COLLATION custom ( + LOCALE = 'fr-x-icu', PROVIDER = 'icu' +); +CREATE TYPE myrange AS range (subtype = text); +CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga); +CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy'); +CREATE TABLE collate_test ( + id integer, + t_en_fr t_en_fr, + d_en_fr d_en_fr, + d_es d_es, + t_en_fr_ga t_en_fr_ga, + d_en_fr_ga d_en_fr_ga, + d_en_fr_ga_arr d_en_fr_ga[], + myrange myrange, + myrange_en_fr_ga myrange_en_fr_ga, + mood mood +); +CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es); +CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr); +CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga); +CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga); +CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr); +CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo'; +CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo'; +CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu"); +CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo'); +CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom; +CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom; +CREATE INDEX icuidx14_myrange ON collate_test(myrange); +CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga); +CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu"; +CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id); +CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1); +CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000); +CREATE INDEX icuidx17_part ON collate_part_1 (val); +-- version of default (and libc) collation is only tracked on glibc systems +SELECT objid::regclass, refobjid::regcollation, +CASE +WHEN refobjversion IS NULL THEN 'no version' +WHEN refobjversion = '' THEN 'unknown version' +ELSE 'some version' END AS version +FROM pg_depend d +LEFT JOIN pg_class c ON c.oid = d.objid +WHERE refclassid = 'pg_collation'::regclass +AND coalesce(relkind, 'i') = 'i' +AND relname LIKE 'icuidx%' +ORDER BY 1, 2; + objid | refobjid | version +---------------------------+------------+-------------- + icuidx01_t_en_fr__d_es | "en-x-icu" | some version + icuidx01_t_en_fr__d_es | "es-x-icu" | some version + icuidx01_t_en_fr__d_es | "fr-x-icu" | some version + icuidx02_d_en_fr | "en-x-icu" | some version + icuidx02_d_en_fr | "fr-x-icu" | some version + icuidx03_t_en_fr_ga | "en-x-icu" | some version + icuidx03_t_en_fr_ga | "fr-x-icu" | some version + icuidx03_t_en_fr_ga | "ga-x-icu" | some version + icuidx04_d_en_fr_ga | "en-x-icu" | some version + icuidx04_d_en_fr_ga | "fr-x-icu" | some version + icuidx04_d_en_fr_ga | "ga-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version + icuidx06_d_en_fr_ga | "default" | no version + icuidx06_d_en_fr_ga | "en-x-icu" | some version + icuidx06_d_en_fr_ga | "fr-x-icu" | some version + icuidx06_d_en_fr_ga | "ga-x-icu" | some version + icuidx07_d_en_fr_ga | "default" | no version + icuidx07_d_en_fr_ga | "en-x-icu" | some version + icuidx07_d_en_fr_ga | "fr-x-icu" | some version + icuidx07_d_en_fr_ga | "ga-x-icu" | some version + icuidx08_d_en_fr_ga | "en-x-icu" | some version + icuidx08_d_en_fr_ga | "fr-x-icu" | some version + icuidx08_d_en_fr_ga | "ga-x-icu" | some version + icuidx09_d_en_fr_ga | "en-x-icu" | some version + icuidx09_d_en_fr_ga | "fr-x-icu" | some version + icuidx09_d_en_fr_ga | "ga-x-icu" | some version + icuidx10_d_en_fr_ga_es | "en-x-icu" | some version + icuidx10_d_en_fr_ga_es | "es-x-icu" | some version + icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version + icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version + icuidx11_d_es | "default" | no version + icuidx11_d_es | "es-x-icu" | some version + icuidx12_custom | "default" | no version + icuidx12_custom | custom | some version + icuidx13_custom | "default" | no version + icuidx13_custom | custom | some version + icuidx14_myrange | "default" | no version + icuidx15_myrange_en_fr_ga | "en-x-icu" | some version + icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version + icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version + icuidx16_mood | "fr-x-icu" | some version + icuidx17_part | "en-x-icu" | some version +(44 rows) + +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text LIKE 'icuidx%' +AND refobjversion IS NOT NULL; +REINDEX TABLE collate_test; +REINDEX TABLE collate_part_0; +REINDEX TABLE collate_part_1; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) + -- cleanup RESET search_path; SET client_min_messages TO warning; diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8_2.out similarity index 89% copy from src/test/regress/expected/collate.icu.utf8.out copy to src/test/regress/expected/collate.icu.utf8_2.out index 60d9263a2f..07e76c4f48 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8_2.out @@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1); t (1 row) +-- collation versioning support +CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu"); +CREATE DOMAIN d_en_fr AS t_en_fr; +CREATE DOMAIN d_es AS text COLLATE "es-x-icu"; +CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu"); +CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga; +CREATE TYPE t_custom AS (meh text, meh2 text); +CREATE DOMAIN d_custom AS t_custom; +CREATE COLLATION custom ( + LOCALE = 'fr-x-icu', PROVIDER = 'icu' +); +CREATE TYPE myrange AS range (subtype = text); +CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga); +CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy'); +CREATE TABLE collate_test ( + id integer, + t_en_fr t_en_fr, + d_en_fr d_en_fr, + d_es d_es, + t_en_fr_ga t_en_fr_ga, + d_en_fr_ga d_en_fr_ga, + d_en_fr_ga_arr d_en_fr_ga[], + myrange myrange, + myrange_en_fr_ga myrange_en_fr_ga, + mood mood +); +CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es); +CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr); +CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga); +CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga); +CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr); +CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo'; +CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo'; +CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu"); +CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo'); +CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom; +CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom; +CREATE INDEX icuidx14_myrange ON collate_test(myrange); +CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga); +CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu"; +CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id); +CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1); +CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000); +CREATE INDEX icuidx17_part ON collate_part_1 (val); +-- version of default (and libc) collation is only tracked on glibc systems +SELECT objid::regclass, refobjid::regcollation, +CASE +WHEN refobjversion IS NULL THEN 'no version' +WHEN refobjversion = '' THEN 'unknown version' +ELSE 'some version' END AS version +FROM pg_depend d +LEFT JOIN pg_class c ON c.oid = d.objid +WHERE refclassid = 'pg_collation'::regclass +AND coalesce(relkind, 'i') = 'i' +AND relname LIKE 'icuidx%' +ORDER BY 1, 2; + objid | refobjid | version +---------------------------+------------+-------------- + icuidx01_t_en_fr__d_es | "en-x-icu" | some version + icuidx01_t_en_fr__d_es | "es-x-icu" | some version + icuidx01_t_en_fr__d_es | "fr-x-icu" | some version + icuidx02_d_en_fr | "en-x-icu" | some version + icuidx02_d_en_fr | "fr-x-icu" | some version + icuidx03_t_en_fr_ga | "en-x-icu" | some version + icuidx03_t_en_fr_ga | "fr-x-icu" | some version + icuidx03_t_en_fr_ga | "ga-x-icu" | some version + icuidx04_d_en_fr_ga | "en-x-icu" | some version + icuidx04_d_en_fr_ga | "fr-x-icu" | some version + icuidx04_d_en_fr_ga | "ga-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version + icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version + icuidx06_d_en_fr_ga | "default" | some version + icuidx06_d_en_fr_ga | "en-x-icu" | some version + icuidx06_d_en_fr_ga | "fr-x-icu" | some version + icuidx06_d_en_fr_ga | "ga-x-icu" | some version + icuidx07_d_en_fr_ga | "default" | some version + icuidx07_d_en_fr_ga | "en-x-icu" | some version + icuidx07_d_en_fr_ga | "fr-x-icu" | some version + icuidx07_d_en_fr_ga | "ga-x-icu" | some version + icuidx08_d_en_fr_ga | "en-x-icu" | some version + icuidx08_d_en_fr_ga | "fr-x-icu" | some version + icuidx08_d_en_fr_ga | "ga-x-icu" | some version + icuidx09_d_en_fr_ga | "en-x-icu" | some version + icuidx09_d_en_fr_ga | "fr-x-icu" | some version + icuidx09_d_en_fr_ga | "ga-x-icu" | some version + icuidx10_d_en_fr_ga_es | "en-x-icu" | some version + icuidx10_d_en_fr_ga_es | "es-x-icu" | some version + icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version + icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version + icuidx11_d_es | "default" | some version + icuidx11_d_es | "es-x-icu" | some version + icuidx12_custom | "default" | some version + icuidx12_custom | custom | some version + icuidx13_custom | "default" | some version + icuidx13_custom | custom | some version + icuidx14_myrange | "default" | some version + icuidx15_myrange_en_fr_ga | "en-x-icu" | some version + icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version + icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version + icuidx16_mood | "fr-x-icu" | some version + icuidx17_part | "en-x-icu" | some version +(44 rows) + +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text LIKE 'icuidx%' +AND refobjversion IS NOT NULL; +REINDEX TABLE collate_test; +REINDEX TABLE collate_part_0; +REINDEX TABLE collate_part_1; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) + -- cleanup RESET search_path; SET client_min_messages TO warning; diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out index 6ddf3a63c3..92dcda5bdc 100644 --- a/src/test/regress/expected/create_index.out +++ b/src/test/regress/expected/create_index.out @@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND obj | objref | deptype ------------------------------------------+------------------------------------------------------------+--------- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i + index concur_reindex_ind2 | collation "default" | n index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a index concur_reindex_ind3 | table concur_reindex_tab | a + index concur_reindex_ind4 | collation "default" | n index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a materialized view concur_reindex_matview | schema public | n table concur_reindex_tab | schema public | n -(9 rows) +(11 rows) REINDEX INDEX CONCURRENTLY concur_reindex_ind1; REINDEX TABLE CONCURRENTLY concur_reindex_tab; @@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND obj | objref | deptype ------------------------------------------+------------------------------------------------------------+--------- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i + index concur_reindex_ind2 | collation "default" | n index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a index concur_reindex_ind3 | table concur_reindex_tab | a + index concur_reindex_ind4 | collation "default" | n index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a materialized view concur_reindex_matview | schema public | n table concur_reindex_tab | schema public | n -(9 rows) +(11 rows) -- Check that comments are preserved CREATE TABLE testcomment (i int); diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql index 35acf91fbf..920a73512f 100644 --- a/src/test/regress/sql/collate.icu.utf8.sql +++ b/src/test/regress/sql/collate.icu.utf8.sql @@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF'); -- they end up in the same partition (but it's platform-dependent which one) SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1); +-- collation versioning support +CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu"); +CREATE DOMAIN d_en_fr AS t_en_fr; +CREATE DOMAIN d_es AS text COLLATE "es-x-icu"; +CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu"); +CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga; +CREATE TYPE t_custom AS (meh text, meh2 text); +CREATE DOMAIN d_custom AS t_custom; + +CREATE COLLATION custom ( + LOCALE = 'fr-x-icu', PROVIDER = 'icu' +); + +CREATE TYPE myrange AS range (subtype = text); +CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga); + +CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy'); + +CREATE TABLE collate_test ( + id integer, + t_en_fr t_en_fr, + d_en_fr d_en_fr, + d_es d_es, + t_en_fr_ga t_en_fr_ga, + d_en_fr_ga d_en_fr_ga, + d_en_fr_ga_arr d_en_fr_ga[], + myrange myrange, + myrange_en_fr_ga myrange_en_fr_ga, + mood mood +); + +CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es); +CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr); +CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga); +CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga); +CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr); +CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo'; +CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo'; +CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz'); +CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu"); +CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo'); +CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom; +CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom; +CREATE INDEX icuidx14_myrange ON collate_test(myrange); +CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga); +CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu"; + +CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id); +CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1); +CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000); +CREATE INDEX icuidx17_part ON collate_part_1 (val); + +-- version of default (and libc) collation is only tracked on glibc systems +SELECT objid::regclass, refobjid::regcollation, +CASE +WHEN refobjversion IS NULL THEN 'no version' +WHEN refobjversion = '' THEN 'unknown version' +ELSE 'some version' END AS version +FROM pg_depend d +LEFT JOIN pg_class c ON c.oid = d.objid +WHERE refclassid = 'pg_collation'::regclass +AND coalesce(relkind, 'i') = 'i' +AND relname LIKE 'icuidx%' +ORDER BY 1, 2; + +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text LIKE 'icuidx%' +AND refobjversion IS NOT NULL; + +REINDEX TABLE collate_test; +REINDEX TABLE collate_part_0; +REINDEX TABLE collate_part_1; + +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; -- cleanup RESET search_path; -- 2.20.1 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename*0="0005-Preserve-index-dependencies-on-collation-during-pg_u-v8."; filename*1=patch From b33dc0533f2285b6e86766b2dc7c1b836b026337 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Thu, 7 Nov 2019 07:33:20 +0100 Subject: [PATCH 5/6] Preserve index dependencies on collation during pg_upgrade A new binary_upgrade_set_index_coll_version() SQL function is added to override the recorded dependency version for all or a single collation, for a specific index. Also teach pg_dump to call this function for all indexes, including indexes created for constraints, when run with --binary-upgrade flag. When pg_upgrade is run against an older version, collation versions are not known and pg_dump will by default emit an alter index command to mark all collation versions as unkown. However, it's possible that pg_upgrade is run without upgrading the underlying collation libraries, so a new option --collation-binary-compatible is added to avoid this behavior. This will result in running pg_dump with a new --unknown-collations-binary-compatible option, that can only be used in binary upgrade mode, to prevent pg_dump from emitting call to this new function if the dependent collation version is unknown. Note that if the collation version is known, this flag won't change the behavior and the previous collation version will be preserved. Author: Julien Rouhaud Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- doc/src/sgml/ref/pgupgrade.sgml | 18 ++ src/backend/catalog/index.c | 62 ++++++ src/backend/commands/tablecmds.c | 52 +++++ src/backend/utils/adt/pg_upgrade_support.c | 25 +++ src/bin/pg_dump/Makefile | 2 + src/bin/pg_dump/pg_backup.h | 1 + src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++- src/bin/pg_dump/pg_dump.h | 3 + src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++----- src/bin/pg_upgrade/dump.c | 4 +- src/bin/pg_upgrade/option.c | 7 + src/bin/pg_upgrade/pg_upgrade.h | 2 + src/include/catalog/dependency.h | 7 + src/include/catalog/index.h | 3 + src/include/catalog/pg_proc.dat | 4 + src/test/perl/PostgresNode.pm | 6 +- 16 files changed, 564 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml index 6629d736b8..b62a1d4eef 100644 --- a/doc/src/sgml/ref/pgupgrade.sgml +++ b/doc/src/sgml/ref/pgupgrade.sgml @@ -212,6 +212,24 @@ + + + + + When upgrading from a PostgreSQL major version 12 or older, or from a + PostgreSQL major version that didn't have support for collation + versioning to a major version that now supports it, all indexes will be + marked as depending on an unknown collation version, as such versions + weren't tracked. As a result, numerous warning messages will be + emitted as it can be a sign of a corrupted index. If you're not + upgrading the collation libraries, and if you're absolutly certain that + all existing indexes are compatible with the current collation + libraries, you can use this flag to change this behavior and mark all + indexes as depending on current collation libraries version. + + + + diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index cd5c93d4ef..b595c90ad0 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3097,6 +3097,68 @@ index_build(Relation heapRelation, SetUserIdAndSecContext(save_userid, save_sec_context); } +static char * +index_force_collation_version(const ObjectAddress *otherObject, + const char *version, + void *userdata) +{ + NewCollationVersionDependency *forced_dependency; + + forced_dependency = (NewCollationVersionDependency *) userdata; + + /* We only care about dependencies on collations. */ + if (otherObject->classId != CollationRelationId) + return NULL; + + /* + * We only care about dependencies on a specific collation if a valid Oid + * was given.= + */ + if (OidIsValid(forced_dependency->oid) && + otherObject->objectId != forced_dependency->oid) + return NULL; + + return forced_dependency->version; +} + +/* index_force_collation_versions + * + * Override collation version dependencies for the given index. If no + * collation identifier is specified, all dependent collation should be + * reset to an unknown version dependency, and no version should be provided + * either. + */ +void +index_force_collation_versions(Oid indexid, Oid coll, char *version) +{ + Relation index; + ObjectAddress object; + NewCollationVersionDependency forced_dependency; + + Assert(version); + + index = relation_open(indexid, AccessExclusiveLock); + + if (index->rd_rel->relkind != RELKIND_INDEX) + ereport(ERROR, + (errcode(ERRCODE_WRONG_OBJECT_TYPE), + (errmsg("\"%s\" is not an index", get_rel_name(indexid))))); + + forced_dependency.oid = InvalidOid; + forced_dependency.version = version; + + object.classId = RelationRelationId; + object.objectId = indexid; + object.objectSubId = 0; + visitDependentObjects(&object, &index_force_collation_version, + &forced_dependency); + + /* Invalidate the index relcache */ + CacheInvalidateRelcache(index); + + relation_close(index, NoLock); +} + /* * IndexCheckExclusion - verify that a new exclusion constraint is satisfied * diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index b7c8d663fc..facda420f2 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -553,6 +553,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx, Relation partitionTbl); static List *GetParentedForeignKeyRefs(Relation partition); static void ATDetachCheckNoForeignKeyRefs(Relation partition); +//static void ATExecDependsOnCollationVersion(Relation rel, List *coll, +// char *version); /* ---------------------------------------------------------------- @@ -3871,6 +3873,11 @@ AlterTableGetLockLevel(List *cmds) cmd_lockmode = AccessShareLock; break; + /* Only used in binary upgrade mode */ + //case AT_DependsOnCollationVersion: + // cmd_lockmode = AccessExclusiveLock; + // break; + default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -4038,6 +4045,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* This command never recurses */ pass = AT_PASS_MISC; break; + //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ... + // * [UNKNOWN VERSION | VERSION ...] */ + // if (!IsBinaryUpgrade) + // ereport(ERROR, + // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM), + // (errmsg("command can only be called when server is in binary upgrade mode")))); + // ATSimplePermissions(rel, ATT_INDEX); + // /* This command never recurses */ + // pass = AT_PASS_MISC; + // break; case AT_SetStorage: /* ALTER COLUMN SET STORAGE */ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE); ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); @@ -4604,6 +4621,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel, Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE); ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name); break; + //case AT_DependsOnCollationVersion: + // /* ATPrepCmd ensured it must be an index */ + // Assert(rel->rd_rel->relkind == RELKIND_INDEX); + // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version); + // break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -17269,3 +17291,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition) table_close(rel, NoLock); } } + +/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ... + * + * A version has to be provided. If the caller wants to notify that the + * collation version to depend on is unknown, an empty string is passed. + */ +//static void +//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version) +//{ +// ObjectAddress object; +// NewCollationVersionDependency forced_dependency; +// +// Assert(version); +// +// if (coll == NIL) +// forced_dependency.oid = InvalidOid; +// else +// forced_dependency.oid = get_collation_oid(coll, false); +// +// forced_dependency.version = version; +// +// object.classId = RelationRelationId; +// object.objectId = rel->rd_id; +// object.objectSubId = 0; +// visitDependentObjects(&object, &index_force_collation_version, +// &forced_dependency); +// +// /* Invalidate the index relcache */ +// CacheInvalidateRelcache(rel); +//} diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c index 0d9e55cdcf..ea7c25ee35 100644 --- a/src/backend/utils/adt/pg_upgrade_support.c +++ b/src/backend/utils/adt/pg_upgrade_support.c @@ -13,6 +13,7 @@ #include "catalog/binary_upgrade.h" #include "catalog/heap.h" +#include "catalog/index.h" #include "catalog/namespace.h" #include "catalog/pg_type.h" #include "commands/extension.h" @@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS) PG_RETURN_VOID(); } + +Datum +binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS) +{ + Oid relid; + Oid coll; + char *version; + + CHECK_IS_BINARY_UPGRADE; + + relid = PG_GETARG_OID(0); + + /* Detect if a collation is specified */ + if (PG_ARGISNULL(1)) + coll = InvalidOid; + else + coll = PG_GETARG_OID(1); + + version = TextDatumGetCString(PG_GETARG_TEXT_PP(2)); + + index_force_collation_versions(relid, coll, version); + + PG_RETURN_VOID(); +} diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile index 2532d9183a..c58ebaa681 100644 --- a/src/bin/pg_dump/Makefile +++ b/src/bin/pg_dump/Makefile @@ -16,6 +16,8 @@ subdir = src/bin/pg_dump top_builddir = ../../.. include $(top_builddir)/src/Makefile.global +export with_icu + override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h index 8c0cedcd98..dc82b076ef 100644 --- a/src/bin/pg_dump/pg_backup.h +++ b/src/bin/pg_dump/pg_backup.h @@ -171,6 +171,7 @@ typedef struct _dumpOptions int sequence_data; /* dump sequence data even in schema-only mode */ int do_nothing; + int unknown_coll_compat; } DumpOptions; /* diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 9aa6496814..a09a0cf53d 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -46,6 +46,7 @@ #include "catalog/pg_attribute_d.h" #include "catalog/pg_cast_d.h" #include "catalog/pg_class_d.h" +#include "catalog/pg_collation_d.h" #include "catalog/pg_default_acl_d.h" #include "catalog/pg_largeobject_d.h" #include "catalog/pg_largeobject_metadata_d.h" @@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer, static const char *getAttrName(int attrnum, TableInfo *tblInfo); static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer); static bool nonemptyReloptions(const char *reloptions); +static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, + int enc, int unknown_coll_compat); static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions, const char *prefix, Archive *fout); static char *get_synchronized_snapshot(Archive *fout); @@ -388,6 +391,7 @@ main(int argc, char **argv) {"no-sync", no_argument, NULL, 7}, {"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1}, {"rows-per-insert", required_argument, NULL, 10}, + {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1}, {NULL, 0, NULL, 0} }; @@ -708,6 +712,10 @@ main(int argc, char **argv) if (archiveFormat != archDirectory && numWorkers > 1) fatal("parallel backup only supported by the directory format"); + /* Unknown collation versions can only be ignored in binary upgrade mode */ + if (dopt.unknown_coll_compat && !dopt.binary_upgrade) + fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode"); + /* Open the output file */ fout = CreateArchive(filename, archiveFormat, compressLevel, dosync, archiveMode, setupDumpWorker); @@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) i_tablespace, i_indreloptions, i_indstatcols, - i_indstatvals; + i_indstatvals, + i_inddependoids, + i_inddependversions; int ntups; for (i = 0; i < numTables; i++) @@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) * is not. */ resetPQExpBuffer(query); - if (fout->remoteVersion >= 110000) + if (fout->remoteVersion >= 130000) + { + appendPQExpBuffer(query, + "SELECT t.tableoid, t.oid, " + "t.relname AS indexname, " + "inh.inhparent AS parentidx, " + "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, " + "i.indnkeyatts AS indnkeyatts, " + "i.indnatts AS indnatts, " + "i.indkey, i.indisclustered, " + "i.indisreplident, " + "c.contype, c.conname, " + "c.condeferrable, c.condeferred, " + "c.tableoid AS contableoid, " + "c.oid AS conoid, " + "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, " + "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, " + "t.reloptions AS indreloptions, " + "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) " + " FROM pg_catalog.pg_attribute " + " WHERE attrelid = i.indexrelid AND " + " attstattarget >= 0) AS indstatcols," + "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) " + " FROM pg_catalog.pg_attribute " + " WHERE attrelid = i.indexrelid AND " + " attstattarget >= 0) AS indstatvals, " + "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) " + " FROM pg_catalog.pg_depend " + " WHERE classid = " CppAsString2(RelationRelationId) " AND " + " objid = i.indexrelid AND " + " objsubid = 0 AND " + " refclassid = " CppAsString2(CollationRelationId) " AND " + " refobjversion IS NOT NULL) AS inddependoids, " + "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) " + " FROM pg_catalog.pg_depend " + " WHERE classid = " CppAsString2(RelationRelationId) " AND " + " objid = i.indexrelid AND " + " objsubid = 0 AND " + " refclassid = " CppAsString2(CollationRelationId) " AND " + " refobjversion IS NOT NULL) AS inddependversions " + "FROM pg_catalog.pg_index i " + "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " + "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) " + "LEFT JOIN pg_catalog.pg_constraint c " + "ON (i.indrelid = c.conrelid AND " + "i.indexrelid = c.conindid AND " + "c.contype IN ('p','u','x')) " + "LEFT JOIN pg_catalog.pg_inherits inh " + "ON (inh.inhrelid = indexrelid) " + "WHERE i.indrelid = '%u'::pg_catalog.oid " + "AND (i.indisvalid OR t2.relkind = 'p') " + "AND i.indisready " + "ORDER BY indexname", + tbinfo->dobj.catId.oid); + } + else if (fout->remoteVersion >= 110000) { appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, " @@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) " " FROM pg_catalog.pg_attribute " " WHERE attrelid = i.indexrelid AND " - " attstattarget >= 0) AS indstatvals " + " attstattarget >= 0) AS indstatvals, " + "' ' AS inddependoids, " + "' ' AS inddependversions " "FROM pg_catalog.pg_index i " "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) " @@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, " "t.reloptions AS indreloptions, " "'' AS indstatcols, " - "'' AS indstatvals " + "'' AS indstatvals, " + "' ' AS inddependoids, " + "' ' AS inddependversions " "FROM pg_catalog.pg_index i " "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " "LEFT JOIN pg_catalog.pg_constraint c " @@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, " "t.reloptions AS indreloptions, " "'' AS indstatcols, " - "'' AS indstatvals " + "'' AS indstatvals, " + "' ' AS inddependoids, " + "' ' AS inddependversions " "FROM pg_catalog.pg_index i " "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " "LEFT JOIN pg_catalog.pg_constraint c " @@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, " "t.reloptions AS indreloptions, " "'' AS indstatcols, " - "'' AS indstatvals " + "'' AS indstatvals, " + "' ' AS inddependoids, " + "' ' AS inddependversions " "FROM pg_catalog.pg_index i " "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " "LEFT JOIN pg_catalog.pg_depend d " @@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, " "null AS indreloptions, " "'' AS indstatcols, " - "'' AS indstatvals " + "'' AS indstatvals, " + "' ' AS inddependoids, " + "' ' AS inddependversions " "FROM pg_catalog.pg_index i " "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) " "LEFT JOIN pg_catalog.pg_depend d " @@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) i_indreloptions = PQfnumber(res, "indreloptions"); i_indstatcols = PQfnumber(res, "indstatcols"); i_indstatvals = PQfnumber(res, "indstatvals"); + i_inddependoids = PQfnumber(res, "inddependoids"); + i_inddependversions = PQfnumber(res, "inddependversions"); tbinfo->indexes = indxinfo = (IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo)); @@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions)); indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols)); indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals)); + indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids)); + indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions)); indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid)); parseOidArray(PQgetvalue(res, j, i_indkey), indxinfo[j].indkeys, indxinfo[j].indnattrs); @@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo) /* * If there's an associated constraint, don't dump the index per se, but - * do dump any comment for it. (This is safe because dependency ordering - * will have ensured the constraint is emitted first.) Note that the - * emitted comment has to be shown as depending on the constraint, not the - * index, in such cases. + * do dump any comment, or in binary upgrade mode dependency on a collation + * version for it. (This is safe because dependency ordering will have + * ensured the constraint is emitted first.) Note that the emitted + * comment has to be shown as depending on the constraint, not the index, + * in such cases. */ if (!is_constraint) { @@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo) } } + if (dopt->binary_upgrade) + appendIndexCollationVersion(q, indxinfo, fout->encoding, + dopt->unknown_coll_compat); + /* If the index defines identity, we need to record that. */ if (indxinfo->indisreplident) { @@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo) if (indstatvalsarray) free(indstatvalsarray); } + else if (dopt->binary_upgrade) + { + appendIndexCollationVersion(q, indxinfo, fout->encoding, + dopt->unknown_coll_compat); + + if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION) + ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId, + ARCHIVE_OPTS(.tag = indxinfo->dobj.name, + .namespace = tbinfo->dobj.namespace->dobj.name, + .tablespace = indxinfo->tablespace, + .owner = tbinfo->rolname, + .description = "INDEX", + .section = SECTION_POST_DATA, + .createStmt = q->data)); + } /* Dump Index Comments */ if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT) @@ -18423,6 +18522,77 @@ nonemptyReloptions(const char *reloptions) return (reloptions != NULL && strlen(reloptions) > 2); } +/* + * Format inddependoids and inddependversions arrays and append it to the given + * buffer in the form of binary_upgrade_set_index_coll_version() calls. + */ +static void +appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc, + int unknown_coll_compat) +{ + char *inddependoids = indxinfo->inddependoids; + char *inddependversions = indxinfo->inddependversions; + char **inddependoidsarray = NULL; + char **inddependversionsarray = NULL; + int ninddependoids; + int ninddependversions; + int i; + + /* + * for older versions that don't record the collation depndency, issue a + * statement to mark the collation version as unknown + */ + if (strcmp(inddependoids, " ") == 0) + { + /* + * do not issue UNKNOWN VERSION is caller specified that those are + * compatible + */ + if (unknown_coll_compat) + return; + + Assert(strcmp(inddependversions, " ") == 0); + + appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n"); + appendPQExpBuffer(buffer, "SELECT " + "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');", + indxinfo->dobj.catId.oid); + return; + } + + parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids); + parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions); + + Assert(ninddependoids == ninddependversions); + + for (i = 0; i < ninddependoids; i++) + { + /* + * If there was an unknown version dependency recorded for this + * collation and the caller asked to mark those as depending ono + * current version, don't emit a binary_upgrade_set_index_coll_version + * function call. + */ + if ((strcmp(inddependversionsarray[i], "''")) == 0 + && unknown_coll_compat) + { + continue; + } + + appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n"); + appendPQExpBuffer(buffer, "SELECT " + "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);", + indxinfo->dobj.catId.oid, + inddependoidsarray[i], + inddependversionsarray[i]); + } + + if (inddependoidsarray) + free(inddependoidsarray); + if (inddependversionsarray) + free(inddependversionsarray); +} + /* * Format a reloptions array and append it to the given buffer. * diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index 21004e5078..6d1df24080 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -364,6 +364,9 @@ typedef struct _indxInfo int indnattrs; /* total number of index attributes */ Oid *indkeys; /* In spite of the name 'indkeys' this field * contains both key and nonkey attributes */ + char *inddependoids; /* oids of collation this index depends on */ + char *inddependversions; /* version of collation this index depends + * on */ bool indisclustered; bool indisreplident; Oid parentidx; /* if partitioned, parent index OID */ diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index 4a9764c2d2..216a704bb7 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -53,6 +53,24 @@ my %pgdump_runs = ( "$tempdir/binary_upgrade.dump", ], }, + binary_coll_compatible => { + dump_cmd => [ + 'pg_dump', + '--no-sync', + '--format=custom', + "--file=$tempdir/binary_coll_compatible.dump", + '-w', + '--schema-only', + '--binary-upgrade', + '--unknown-collations-binary-compatible', + '-d', 'postgres', # alternative way to specify database + ], + restore_cmd => [ + 'pg_restore', '-Fc', '--verbose', + "--file=$tempdir/binary_coll_compatible.sql", + "$tempdir/binary_coll_compatible.dump", + ], + }, clean => { dump_cmd => [ 'pg_dump', @@ -387,6 +405,7 @@ my %dump_test_schema_runs = ( # are flags used to exclude specific items (ACLs, blobs, etc). my %full_runs = ( binary_upgrade => 1, + binary_coll_compatible => 1, clean => 1, clean_if_exists => 1, createdb => 1, @@ -914,9 +933,10 @@ my %tests = ( test_schema_plus_blobs => 1, }, unlike => { - binary_upgrade => 1, - no_blobs => 1, - schema_only => 1, + binary_upgrade => 1, + binary_coll_compatible => 1, + no_blobs => 1, + schema_only => 1, }, }, @@ -1178,6 +1198,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, exclude_test_table => 1, exclude_test_table_data => 1, @@ -1203,6 +1224,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -1238,6 +1260,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -1260,6 +1283,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -1281,6 +1305,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -1302,6 +1327,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -1665,6 +1691,7 @@ my %tests = ( { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, }, }, @@ -1679,7 +1706,7 @@ my %tests = ( \n.*^ \QALTER TYPE dump_test.planets ADD VALUE 'mars';\E \n/xms, - like => { binary_upgrade => 1, }, + like => { binary_upgrade => 1, binary_coll_compatible => 1, }, }, 'CREATE TYPE dump_test.textrange AS RANGE' => { @@ -2347,6 +2374,7 @@ my %tests = ( { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, }, }, @@ -2540,6 +2568,7 @@ my %tests = ( }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, }, }, @@ -2607,6 +2636,7 @@ my %tests = ( /xm, like => { binary_upgrade => 1, + binary_coll_compatible => 1, clean => 1, clean_if_exists => 1, createdb => 1, @@ -2678,6 +2708,7 @@ my %tests = ( /xm, like => { binary_upgrade => 1, + binary_coll_compatible => 1, clean => 1, clean_if_exists => 1, createdb => 1, @@ -3145,6 +3176,7 @@ my %tests = ( { %full_runs, %dump_test_schema_runs, section_post_data => 1, }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -3160,6 +3192,7 @@ my %tests = ( { %full_runs, %dump_test_schema_runs, section_post_data => 1, }, unlike => { binary_upgrade => 1, + binary_coll_compatible => 1, exclude_dump_test_schema => 1, schema_only => 1, }, @@ -3292,16 +3325,53 @@ my %tests = ( %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1 }, + }, + + "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => { + create_order => 101, + create_sql => ' + CREATE TABLE dump_test.regress_table_coll(id integer, val text); + CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu"); + UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';', + regexp => qr/^ + \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n + \n + \Q-- For binary upgrade, restore dependent collation version.\E\n + \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm, + like => { binary_upgrade => 1, binary_coll_compatible => 1, }, + icu => 1, + }, + "binary_upgrade_set_index_coll_version(?, ?, '')" => { + create_order => 102, + create_sql => ' + CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text); + CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu"); + UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';', + regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/, + like => { binary_upgrade => 1}, + # should not appear in binary_coll_compatible case! + unlike => { binary_coll_compatible => 1}, + icu => 1, }); ######################################### # Create a PG instance to test actually dumping from -my $node = get_new_node('main'); -$node->init; -$node->start; +my $main_node = get_new_node('main'); +$main_node->init; +$main_node->start; + +my $port = $main_node->port; + +# And another instance to validate the binary dump +my $bin_node = get_new_node('binary'); +$bin_node->init; +$bin_node->start; -my $port = $node->port; +my $bin_port = $bin_node->port; + +# and add a $node variable pointing to main_node for now +my $node = $main_node; # We need to see if this system supports CREATE COLLATION or not # If it doesn't then we will skip all the COLLATION-related tests. @@ -3325,6 +3395,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;'); # command_fails_like is actually 2 tests) my $num_tests = 12; +# 4 more tests for restoring globals and binary_upgrade dump, dumping it again +# and regenerating the sql file +$num_tests+= 4; + foreach my $run (sort keys %pgdump_runs) { my $test_key = $run; @@ -3375,16 +3449,29 @@ foreach my $run (sort keys %pgdump_runs) next; } + # Skip any icu-related commands if there is no icu support + if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu})) + { + next; + } + # If there is a like entry, but no unlike entry, then we will test the like case if ($tests{$test}->{like}->{$test_key} && !defined($tests{$test}->{unlike}->{$test_key})) { $num_tests++; + + # binary_upgrade tests are also run after being restored and + # re-dumped. + $num_tests++ if ($test_key eq 'binary_upgrade'); } else { # We will test everything that isn't a 'like' $num_tests++; + # binary_upgrade tests are also run after being restored and + # re-dumped. + $num_tests++ if ($test_key eq 'binary_upgrade'); } } } @@ -3432,6 +3519,12 @@ foreach my $test ( next; } + # Skip any icu-related commands if there is no icu support + if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu})) + { + next; + } + # Add terminating semicolon $create_sql{$test_db} .= $tests{$test}->{create_sql} . ";"; } @@ -3485,79 +3578,116 @@ command_fails_like( ######################################### # Run all runs -foreach my $run (sort keys %pgdump_runs) +foreach my $pass (1, 2) { - my $test_key = $run; - my $run_db = 'postgres'; - - $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} }, - "$run: pg_dump runs"); - - if ($pgdump_runs{$run}->{restore_cmd}) + foreach my $run (sort keys %pgdump_runs) { - $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} }, - "$run: pg_restore runs"); - } - - if ($pgdump_runs{$run}->{test_key}) - { - $test_key = $pgdump_runs{$run}->{test_key}; - } - - my $output_file = slurp_file("$tempdir/${run}.sql"); + my $test_key = $run; + my $run_db = 'postgres'; - ######################################### - # Run all tests where this run is included - # as either a 'like' or 'unlike' test. + # we only test binary upgrade on the 2nd pass + next if ($pass == 2 and $test_key ne 'binary_upgrade'); - foreach my $test (sort keys %tests) - { - my $test_db = 'postgres'; + $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} }, + "$run: pg_dump runs"); - if (defined($pgdump_runs{$run}->{database})) + if ($pgdump_runs{$run}->{restore_cmd}) { - $run_db = $pgdump_runs{$run}->{database}; + $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} }, + "$run: pg_restore runs"); } - if (defined($tests{$test}->{database})) + if ($pgdump_runs{$run}->{test_key}) { - $test_db = $tests{$test}->{database}; + $test_key = $pgdump_runs{$run}->{test_key}; } - # Skip any collation-related commands if there is no collation support - if (!$collation_support && defined($tests{$test}->{collation})) - { - next; - } + my $output_file = slurp_file("$tempdir/${run}.sql"); - if ($run_db ne $test_db) - { - next; - } + ######################################### + # Run all tests where this run is included + # as either a 'like' or 'unlike' test. - # Run the test listed as a like, unless it is specifically noted - # as an unlike (generally due to an explicit exclusion or similar). - if ($tests{$test}->{like}->{$test_key} - && !defined($tests{$test}->{unlike}->{$test_key})) + foreach my $test (sort keys %tests) { - if (!ok($output_file =~ $tests{$test}->{regexp}, - "$run: should dump $test")) + my $test_db = 'postgres'; + + if (defined($pgdump_runs{$run}->{database})) { - diag("Review $run results in $tempdir"); + $run_db = $pgdump_runs{$run}->{database}; } - } - else - { - if (!ok($output_file !~ $tests{$test}->{regexp}, - "$run: should not dump $test")) + + if (defined($tests{$test}->{database})) + { + $test_db = $tests{$test}->{database}; + } + + # Skip any collation-related commands if there is no collation support + if (!$collation_support && defined($tests{$test}->{collation})) + { + next; + } + + # Skip any icu-related commands if there is no icu support + if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu})) + { + next; + } + + if ($run_db ne $test_db) + { + next; + } + + # Run the test listed as a like, unless it is specifically noted + # as an unlike (generally due to an explicit exclusion or similar). + if ($tests{$test}->{like}->{$test_key} + && !defined($tests{$test}->{unlike}->{$test_key})) { - diag("Review $run results in $tempdir"); + if (!ok($output_file =~ $tests{$test}->{regexp}, + "$run: should dump $test")) + { + diag("Review $run results in $tempdir"); + } + } + else + { + if (!ok($output_file !~ $tests{$test}->{regexp}, + "$run: should not dump $test")) + { + diag("Review $run results in $tempdir"); + } } } } + + # After all dump have been generated, restore the binary_upgrade dump with + # the required global objects on a suitable node, and continue with the 2nd + # pass. + if ($pass == 1) + { + # Stop the original database instance as we don't need it anymore. + $node->stop('fast'); + + $bin_node->command_ok(\@{['psql', + "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]}, + "Restore globals"); + + $bin_node->stop('fast'); + $bin_node->start(binary_start => 1); + $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port, + '-d', 'postgres', + "$tempdir/binary_upgrade.dump"]}, + "Restore the binary_upgrade dump"); + $bin_node->stop('fast'); + $bin_node->start; + + # And change $node to point to the freshly restored node. + $node = $bin_node; + } } ######################################### # Stop the database instance, which will be removed at the end of the tests. -$node->stop('fast'); +$bin_node->stop('fast'); diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c index 4d730adfe2..672ecda169 100644 --- a/src/bin/pg_upgrade/dump.c +++ b/src/bin/pg_upgrade/dump.c @@ -52,9 +52,11 @@ generate_old_dump(void) parallel_exec_prog(log_file_name, NULL, "\"%s/pg_dump\" %s --schema-only --quote-all-identifiers " - "--binary-upgrade --format=custom %s --file=\"%s\" %s", + "--binary-upgrade --format=custom %s %s --file=\"%s\" %s", new_cluster.bindir, cluster_conn_opts(&old_cluster), log_opts.verbose ? "--verbose" : "", + user_opts.coll_compat ? + "--unknown-collations-binary-compatible" : "", sql_file_name, escaped_connstr.data); termPQExpBuffer(&escaped_connstr); diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c index eed70fff4f..d36d6a283f 100644 --- a/src/bin/pg_upgrade/option.c +++ b/src/bin/pg_upgrade/option.c @@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[]) {"socketdir", required_argument, NULL, 's'}, {"verbose", no_argument, NULL, 'v'}, {"clone", no_argument, NULL, 1}, + {"collation-binary-compatible", no_argument, NULL, 2}, {NULL, 0, NULL, 0} }; @@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[]) user_opts.transfer_mode = TRANSFER_MODE_CLONE; break; + case 2: + user_opts.coll_compat = true; + break; + default: fprintf(stderr, _("Try \"%s --help\" for more information.\n"), os_info.progname); @@ -307,6 +312,8 @@ usage(void) printf(_(" -v, --verbose enable verbose internal logging\n")); printf(_(" -V, --version display version information, then exit\n")); printf(_(" --clone clone instead of copying files to new cluster\n")); + printf(_(" --collation-binary-compatible mark collations as depending on current collation\n" + " versions rather than unknown if they're unknown\n")); printf(_(" -?, --help show this help, then exit\n")); printf(_("\n" "Before running pg_upgrade you must:\n" diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h index b156b516cc..af34dff920 100644 --- a/src/bin/pg_upgrade/pg_upgrade.h +++ b/src/bin/pg_upgrade/pg_upgrade.h @@ -294,6 +294,8 @@ typedef struct transferMode transfer_mode; /* copy files or link them? */ int jobs; /* number of processes/threads to use */ char *socketdir; /* directory to use for Unix sockets */ + bool coll_compat; /* should we skip marking index collations as + * unknown version */ } UserOpts; typedef struct diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h index 6de17e25e8..dc99b49b8e 100644 --- a/src/include/catalog/dependency.h +++ b/src/include/catalog/dependency.h @@ -130,6 +130,13 @@ typedef enum ObjectClass #define LAST_OCLASS OCLASS_TRANSFORM +/* Struct describing one forced collation version dependency */ +typedef struct NewCollationVersionDependency +{ + char *version; /* forced collation version */ + Oid oid; /* target collation oid */ +} NewCollationVersionDependency; + /* flag bits for performDeletion/performMultipleDeletions: */ #define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */ #define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */ diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index c619d02465..69d163f3cc 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo, extern void index_check_collation_versions(Oid relid); +extern void index_force_collation_versions(Oid indexid, Oid coll, + char *version); + extern void index_build(Relation heapRelation, Relation indexRelation, IndexInfo *indexInfo, diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index f6503428cc..bca564644c 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10195,6 +10195,10 @@ proname => 'binary_upgrade_set_missing_value', provolatile => 'v', proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text', prosrc => 'binary_upgrade_set_missing_value' }, +{ oid => '8178', descr => 'for use by pg_upgrade', + proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v', + proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text', + prosrc => 'binary_upgrade_set_index_coll_version' }, # conversion functions { oid => '4302', diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm index 9575268bd7..bba07a2319 100644 --- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -764,10 +764,14 @@ sub start local %ENV = %ENV; delete $ENV{PGAPPNAME}; + my $options = "--cluster-name=$name"; + + $options .= ' -b' if ($params{binary_start}); + # Note: We set the cluster_name here, not in postgresql.conf (in # sub init) so that it does not get copied to standbys. $ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l', - $self->logfile, '-o', "--cluster-name=$name", 'start'); + $self->logfile, '-o', $options, 'start'); } if ($ret != 0) -- 2.20.1 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename*0="0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v8."; filename*1=patch From b9d80e5eff355e930a173c6cf8b03e3a0d623c57 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Wed, 11 Dec 2019 13:54:32 +0100 Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH VERSION This command allows privileged users to specifify that the currently installed collation library version, for a specific collation, is binary compatible with the one that was installed when the specified index was built for the last time. Author: Julien Rouhaud Reviewed-by: Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com --- doc/src/sgml/ref/alter_index.sgml | 13 +++ src/backend/catalog/index.c | 2 +- src/backend/commands/tablecmds.c | 90 +++++++++---------- src/backend/nodes/copyfuncs.c | 1 + src/backend/parser/gram.y | 8 ++ src/bin/psql/tab-complete.c | 26 +++++- src/include/catalog/index.h | 3 + src/include/nodes/parsenodes.h | 4 +- .../regress/expected/collate.icu.utf8.out | 17 ++++ .../regress/expected/collate.icu.utf8_2.out | 17 ++++ src/test/regress/sql/collate.icu.utf8.sql | 9 ++ 11 files changed, 139 insertions(+), 51 deletions(-) diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml index 6d34dbb74e..8661b031e9 100644 --- a/doc/src/sgml/ref/alter_index.sgml +++ b/doc/src/sgml/ref/alter_index.sgml @@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] name RENA ALTER INDEX [ IF EXISTS ] name SET TABLESPACE tablespace_name ALTER INDEX name ATTACH PARTITION index_name ALTER INDEX name DEPENDS ON EXTENSION extension_name +ALTER INDEX name ALTER COLLATION collation_name REFRESH VERSION ALTER INDEX [ IF EXISTS ] name SET ( storage_parameter = value [, ... ] ) ALTER INDEX [ IF EXISTS ] name RESET ( storage_parameter [, ... ] ) ALTER INDEX [ IF EXISTS ] name ALTER [ COLUMN ] column_number @@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE name + + ALTER COLLATION + + + This form update the index existing dependency on a specific collation, + to specificy the the currently installed collation version is compatible + with the version used the last time the index was built. Be aware that + an incorrect use of this form can hide a corruption on the index. + + + + SET ( storage_parameter = value [, ... ] ) diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index b595c90ad0..a92977fef7 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -3097,7 +3097,7 @@ index_build(Relation heapRelation, SetUserIdAndSecContext(save_userid, save_sec_context); } -static char * +char * index_force_collation_version(const ObjectAddress *otherObject, const char *version, void *userdata) diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index facda420f2..feaea2024e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -93,6 +93,7 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/partcache.h" +#include "utils/pg_locale.h" #include "utils/relcache.h" #include "utils/ruleutils.h" #include "utils/snapmgr.h" @@ -553,8 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx, Relation partitionTbl); static List *GetParentedForeignKeyRefs(Relation partition); static void ATDetachCheckNoForeignKeyRefs(Relation partition); -//static void ATExecDependsOnCollationVersion(Relation rel, List *coll, -// char *version); +static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll); /* ---------------------------------------------------------------- @@ -3873,10 +3873,9 @@ AlterTableGetLockLevel(List *cmds) cmd_lockmode = AccessShareLock; break; - /* Only used in binary upgrade mode */ - //case AT_DependsOnCollationVersion: - // cmd_lockmode = AccessExclusiveLock; - // break; + case AT_AlterCollationRefreshVersion: + cmd_lockmode = AccessExclusiveLock; + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", @@ -4045,16 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, /* This command never recurses */ pass = AT_PASS_MISC; break; - //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ... - // * [UNKNOWN VERSION | VERSION ...] */ - // if (!IsBinaryUpgrade) - // ereport(ERROR, - // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM), - // (errmsg("command can only be called when server is in binary upgrade mode")))); - // ATSimplePermissions(rel, ATT_INDEX); - // /* This command never recurses */ - // pass = AT_PASS_MISC; - // break; + case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... + * REFRESH VERSION */ + ATSimplePermissions(rel, ATT_INDEX); + /* This command never recurses */ + pass = AT_PASS_MISC; + break; case AT_SetStorage: /* ALTER COLUMN SET STORAGE */ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE); ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); @@ -4621,11 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel, Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE); ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name); break; - //case AT_DependsOnCollationVersion: - // /* ATPrepCmd ensured it must be an index */ - // Assert(rel->rd_rel->relkind == RELKIND_INDEX); - // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version); - // break; + case AT_AlterCollationRefreshVersion: + /* ATPrepCmd ensured it must be an index */ + Assert(rel->rd_rel->relkind == RELKIND_INDEX); + ATExecAlterCollationRefreshVersion(rel, cmd->object); + break; default: /* oops */ elog(ERROR, "unrecognized alter table type: %d", (int) cmd->subtype); @@ -17292,32 +17287,31 @@ ATDetachCheckNoForeignKeyRefs(Relation partition) } } -/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ... +/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION * - * A version has to be provided. If the caller wants to notify that the - * collation version to depend on is unknown, an empty string is passed. + * This override an existing dependency on a specific collation for a specific + * index to depend on the current collation version. */ -//static void -//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version) -//{ -// ObjectAddress object; -// NewCollationVersionDependency forced_dependency; -// -// Assert(version); -// -// if (coll == NIL) -// forced_dependency.oid = InvalidOid; -// else -// forced_dependency.oid = get_collation_oid(coll, false); -// -// forced_dependency.version = version; -// -// object.classId = RelationRelationId; -// object.objectId = rel->rd_id; -// object.objectSubId = 0; -// visitDependentObjects(&object, &index_force_collation_version, -// &forced_dependency); -// -// /* Invalidate the index relcache */ -// CacheInvalidateRelcache(rel); -//} +static void +ATExecAlterCollationRefreshVersion(Relation rel, List *coll) +{ + ObjectAddress object; + NewCollationVersionDependency forced_dependency; + + Assert(coll != NIL); + forced_dependency.oid = get_collation_oid(coll, false); + + /* Retrieve the current version for the CURRENT VERSION case. */ + Assert(OidIsValid(forced_dependency.oid)); + forced_dependency.version = + get_collation_version_for_oid(forced_dependency.oid); + + object.classId = RelationRelationId; + object.objectId = rel->rd_id; + object.objectSubId = 0; + visitDependentObjects(&object, &index_force_collation_version, + &forced_dependency); + + /* Invalidate the index relcache */ + CacheInvalidateRelcache(rel); +} diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index bf4f793ba2..57c87ded58 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -3174,6 +3174,7 @@ _copyAlterTableCmd(const AlterTableCmd *from) COPY_SCALAR_FIELD(subtype); COPY_STRING_FIELD(name); + COPY_NODE_FIELD(object); COPY_SCALAR_FIELD(num); COPY_NODE_FIELD(newowner); COPY_NODE_FIELD(def); diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index b6d6a0e239..886d065a8e 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2569,6 +2569,14 @@ alter_table_cmd: n->subtype = AT_NoForceRowSecurity; $$ = (Node *)n; } + /* ALTER INDEX ALTER COLLATION ... REFRESH VERSION */ + | ALTER COLLATION any_name REFRESH VERSION_P + { + AlterTableCmd *n = makeNode(AlterTableCmd); + n->subtype = AT_AlterCollationRefreshVersion; + n->object = $3; + $$ = (Node *)n; + } | alter_generic_options { AlterTableCmd *n = makeNode(AlterTableCmd); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index dc03fbde13..4cc1646763 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -45,6 +45,7 @@ #include "catalog/pg_am_d.h" #include "catalog/pg_class_d.h" +#include "catalog/pg_collation_d.h" #include "common.h" #include "libpq-fe.h" #include "pqexpbuffer.h" @@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = { " (SELECT tgrelid FROM pg_catalog.pg_trigger "\ " WHERE pg_catalog.quote_ident(tgname)='%s')" +/* the silly-looking length condition is just to eat up the current word */ +#define Query_for_list_of_colls_for_one_index \ +" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \ +" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \ +" pg_catalog.pg_class c" \ +" WHERE (%d = pg_catalog.length('%s'))" \ +" AND d.refclassid = " CppAsString2(CollationRelationId) \ +" AND d.refobjid = coll.oid " \ +" AND d.classid = " CppAsString2(RelationRelationId) \ +" AND d.objid = c.oid " \ +" AND c.relkind = " CppAsString2(RELKIND_INDEX) \ +" AND pg_catalog.pg_table_is_visible(c.oid) " \ +" AND c.relname = '%s'" + #define Query_for_list_of_ts_configurations \ "SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\ " WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'" @@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end) /* ALTER INDEX */ else if (Matches("ALTER", "INDEX", MatchAny)) COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET", - "RESET", "ATTACH PARTITION"); + "RESET", "ATTACH PARTITION", "ALTER COLLATION"); else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH")) COMPLETE_WITH("PARTITION"); else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION")) @@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end) "buffering =", /* GiST */ "pages_per_range =", "autosummarize =" /* BRIN */ ); + /* ALTER INDEX ALTER COLLATION */ + else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION")) + { + completion_info_charp = prev4_wd; + COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index); + } + /* ALTER INDEX ALTER COLLATION */ + else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny)) + COMPLETE_WITH("REFRESH VERSION"); /* ALTER LANGUAGE */ else if (Matches("ALTER", "LANGUAGE", MatchAny)) diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index 69d163f3cc..065488374e 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo, extern void index_check_collation_versions(Oid relid); +extern char *index_force_collation_version(const ObjectAddress *otherObject, + const char *version, + void *userdata); extern void index_force_collation_versions(Oid indexid, Oid coll, char *version); diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index c96d027362..9f637597a7 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -1844,7 +1844,8 @@ typedef enum AlterTableType AT_DetachPartition, /* DETACH PARTITION */ AT_AddIdentity, /* ADD IDENTITY */ AT_SetIdentity, /* SET identity column options */ - AT_DropIdentity /* DROP IDENTITY */ + AT_DropIdentity, /* DROP IDENTITY */ + AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */ } AlterTableType; typedef struct ReplicaIdentityStmt @@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */ AlterTableType subtype; /* Type of table alteration to apply */ char *name; /* column, constraint, or trigger to act on, * or tablespace */ + List *object; /* collation to act on if it's a collation */ int16 num; /* attribute number for columns referenced by * number */ RoleSpec *newowner; diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out index f049189897..2902160688 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8.out @@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; ------- (0 rows) +-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text = 'icuidx17_part' +AND refobjversion IS NOT NULL; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +--------------- + icuidx17_part +(1 row) + +ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) + -- cleanup RESET search_path; SET client_min_messages TO warning; diff --git a/src/test/regress/expected/collate.icu.utf8_2.out b/src/test/regress/expected/collate.icu.utf8_2.out index 07e76c4f48..33a56af104 100644 --- a/src/test/regress/expected/collate.icu.utf8_2.out +++ b/src/test/regress/expected/collate.icu.utf8_2.out @@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; ------- (0 rows) +-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text = 'icuidx17_part' +AND refobjversion IS NOT NULL; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +--------------- + icuidx17_part +(1 row) + +ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) + -- cleanup RESET search_path; SET client_min_messages TO warning; diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql index 920a73512f..10329e3498 100644 --- a/src/test/regress/sql/collate.icu.utf8.sql +++ b/src/test/regress/sql/collate.icu.utf8.sql @@ -793,6 +793,15 @@ REINDEX TABLE collate_part_1; SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; +-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text = 'icuidx17_part' +AND refobjversion IS NOT NULL; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; +ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + -- cleanup RESET search_path; SET client_min_messages TO warning; -- 2.20.1 --6c2NcOVqGQ03X4Wi--