agora inbox for [email protected]
help / color / mirror / Atom feed[sqlsmith] Crash in gather_readnext
76+ messages / 5 participants
[nested] [flat]
* [sqlsmith] Crash in gather_readnext
@ 2016-12-05 20:07 Andreas Seltenreich <[email protected]>
2016-12-05 21:01 ` Re: [sqlsmith] Crash in gather_readnext Robert Haas <[email protected]>
0 siblings, 1 reply; 76+ messages in thread
From: Andreas Seltenreich @ 2016-12-05 20:07 UTC (permalink / raw)
To: pgsql-hackers
Hi,
on master as of a0ae54d, there's a 1 in 10e6 chance sqlsmith catches
gather_readnext reading beyond the gatherstate->readers array with
readers[gatherstate->readnext]. Sample backtrace below.
As readnext is never explicitly initialized, I think what happens is
that a rescan gets less workers than the initial scan, and the dangling
readnext points outside the array. I'm no longer seeing these crashes
when explicitly initializing readnext to 0 like in the attached patch.
regards,
Andreas
Program terminated with signal SIGSEGV, Segmentation fault.
#0 shm_mq_receive (mqh=0x259, nbytesp=nbytesp@entry=0x7ffc55ce0580, datap=datap@entry=0x7ffc55ce0588, nowait=nowait@entry=1 '\001') at shm_mq.c:520
520 shm_mq *mq = mqh->mqh_queue;
(gdb) bt
#0 shm_mq_receive (mqh=0x259, nbytesp=nbytesp@entry=0x7ffc55ce0580, datap=datap@entry=0x7ffc55ce0588, nowait=nowait@entry=1 '\001') at shm_mq.c:520
#1 0x000000000060b8b7 in TupleQueueReaderNext (reader=reader@entry=0x5446c10, nowait=nowait@entry=1 '\001', done=done@entry=0x7ffc55ce065b "") at tqueue.c:692
#2 0x00000000005f5e03 in gather_readnext (gatherstate=0x52a9918) at nodeGather.c:339
#3 gather_getnext (gatherstate=0x52a9918) at nodeGather.c:292
#4 ExecGather (node=node@entry=0x52a9918) at nodeGather.c:233
#5 0x00000000005e3b68 in ExecProcNode (node=0x52a9918) at execProcnode.c:515
#6 0x00000000005eb2f2 in ExecScanFetch (recheckMtd=0x605e40 <SubqueryRecheck>, accessMtd=0x605e50 <SubqueryNext>, node=0x52a86c0) at execScan.c:95
#7 ExecScan (node=node@entry=0x52a86c0, accessMtd=accessMtd@entry=0x605e50 <SubqueryNext>, recheckMtd=recheckMtd@entry=0x605e40 <SubqueryRecheck>) at execScan.c:180
#8 0x0000000000605e6f in ExecSubqueryScan (node=node@entry=0x52a86c0) at nodeSubqueryscan.c:85
#9 0x00000000005e3c68 in ExecProcNode (node=node@entry=0x52a86c0) at execProcnode.c:445
#10 0x00000000006001d6 in ExecNestLoop (node=node@entry=0x52a7978) at nodeNestloop.c:123
#11 0x00000000005e3bf8 in ExecProcNode (node=node@entry=0x52a7978) at execProcnode.c:476
#12 0x00000000006001d6 in ExecNestLoop (node=node@entry=0x52a5120) at nodeNestloop.c:123
#13 0x00000000005e3bf8 in ExecProcNode (node=node@entry=0x52a5120) at execProcnode.c:476
#14 0x00000000006001d6 in ExecNestLoop (node=node@entry=0x52a3d50) at nodeNestloop.c:123
#15 0x00000000005e3bf8 in ExecProcNode (node=0x52a3d50) at execProcnode.c:476
#16 0x00000000006015e5 in ExecResult (node=node@entry=0x52a3140) at nodeResult.c:130
#17 0x00000000005e3d18 in ExecProcNode (node=node@entry=0x52a3140) at execProcnode.c:392
#18 0x00000000005fb360 in ExecLimit (node=node@entry=0x52a2e70) at nodeLimit.c:91
#19 0x00000000005e3af8 in ExecProcNode (node=node@entry=0x52a2e70) at execProcnode.c:531
#20 0x0000000000600299 in ExecNestLoop (node=node@entry=0x52a1a10) at nodeNestloop.c:174
#21 0x00000000005e3bf8 in ExecProcNode (node=node@entry=0x52a1a10) at execProcnode.c:476
#22 0x00000000006001d6 in ExecNestLoop (node=node@entry=0x52a16d0) at nodeNestloop.c:123
#23 0x00000000005e3bf8 in ExecProcNode (node=node@entry=0x52a16d0) at execProcnode.c:476
#24 0x00000000005dfdae in ExecutePlan (dest=0x50cbb00, direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x52a16d0, estate=0x3610968) at execMain.c:1567
#25 standard_ExecutorRun (queryDesc=0x36805b8, direction=<optimized out>, count=0) at execMain.c:338
#26 0x0000000000701a58 in PortalRunSelect (portal=portal@entry=0x529da38, forward=forward@entry=1 '\001', count=0, count@entry=9223372036854775807, dest=dest@entry=0x50cbb00) at pquery.c:946
#27 0x000000000070300e in PortalRun (portal=portal@entry=0x529da38, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1 '\001', dest=dest@entry=0x50cbb00, altdest=altdest@entry=0x50cbb00, completionTag=completionTag@entry=0x7ffc55ce0ed0 "") at pquery.c:787
#28 0x0000000000700869 in exec_simple_query (query_string=0x45d3028 "select ...") at postgres.c:1094
#29 PostgresMain (argc=<optimized out>, argv=argv@entry=0x23ce878, dbname=<optimized out>, username=<optimized out>) at postgres.c:4069
#30 0x000000000046d9d9 in BackendRun (port=0x23d1ad0) at postmaster.c:4271
#31 BackendStartup (port=0x23d1ad0) at postmaster.c:3945
#32 ServerLoop () at postmaster.c:1701
#33 0x0000000000698ed9 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x23a05c0) at postmaster.c:1309
#34 0x000000000046ebbd in main (argc=4, argv=0x23a05c0) at main.c:228
^ permalink raw reply [nested|flat] 76+ messages in thread
* Re: [sqlsmith] Crash in gather_readnext
2016-12-05 20:07 [sqlsmith] Crash in gather_readnext Andreas Seltenreich <[email protected]>
@ 2016-12-05 21:01 ` Robert Haas <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Robert Haas @ 2016-12-05 21:01 UTC (permalink / raw)
To: Andreas Seltenreich <[email protected]>; +Cc: pgsql-hackers
On Mon, Dec 5, 2016 at 3:07 PM, Andreas Seltenreich <[email protected]> wrote:
> on master as of a0ae54d, there's a 1 in 10e6 chance sqlsmith catches
> gather_readnext reading beyond the gatherstate->readers array with
> readers[gatherstate->readnext]. Sample backtrace below.
>
> As readnext is never explicitly initialized, I think what happens is
> that a rescan gets less workers than the initial scan, and the dangling
> readnext points outside the array. I'm no longer seeing these crashes
> when explicitly initializing readnext to 0 like in the attached patch.
Thanks, great catch! Committed and back-patched to 9.6.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Michael Paquier, Mark Dilger
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/access/index/indexam.c | 59 ++++++++++++++
src/backend/catalog/index.c | 79 ++++++++++++++++++-
src/backend/commands/indexcmds.c | 37 ++++++++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 47 +++++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/access/genam.h | 1 +
src/include/catalog/index.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
.../regress/expected/collate.icu.utf8.out | 12 ++-
src/test/regress/expected/create_index.out | 27 +++++++
src/test/regress/sql/collate.icu.utf8.sql | 12 ++-
src/test/regress/sql/create_index.sql | 18 +++++
15 files changed, 301 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..c4749c338b 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ dependency handled currently is the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 3d2dbed708..dc1c85cf0d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -145,6 +145,65 @@ index_open(Oid relationId, LOCKMODE lockmode)
return r;
}
+/* ----------------
+ * try_index_open - open an index relation by relation OID
+ *
+ * Same as index_open, except return NULL instead of failing
+ * if the index does not exist.
+ * ----------------
+ */
+Relation
+try_index_open(Oid relationId, LOCKMODE lockmode)
+{
+ Relation r;
+
+ Assert(lockmode >= NoLock && lockmode < MAX_LOCKMODES);
+
+ /* Get the lock first */
+ if (lockmode != NoLock)
+ LockRelationOid(relationId, lockmode);
+
+ /*
+ * Now that we have the lock, probe to see if the relation really exists
+ * or not.
+ */
+ if (!SearchSysCacheExists1(RELOID, ObjectIdGetDatum(relationId)))
+ {
+ /* Release useless lock */
+ if (lockmode != NoLock)
+ UnlockRelationOid(relationId, lockmode);
+
+ return NULL;
+ }
+
+ /* Should be safe to do a relcache load */
+ r = RelationIdGetRelation(relationId);
+
+ if (!RelationIsValid(r))
+ elog(ERROR, "could not open relation with OID %u", relationId);
+
+ /* If we didn't get the lock ourselves, assert that caller holds one */
+ Assert(lockmode != NoLock ||
+ CheckRelationLockedByMe(r, AccessShareLock, true));
+
+ if (r->rd_rel->relkind != RELKIND_INDEX &&
+ r->rd_rel->relkind != RELKIND_PARTITIONED_INDEX)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not an index",
+ RelationGetRelationName(r))));
+ }
+
+ /* Make note that we've accessed a temporary relation */
+ if (RelationUsesLocalBuffers(r))
+ MyXactFlags |= XACT_FLAGS_ACCESSEDTEMPNAMESPACE;
+
+ pgstat_initstats(r);
+
+ return r;
+}
+
/* ----------------
* index_close - close an index relation
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..47e6a54149 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprecated collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,77 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4068,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..90ea95f40d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -2605,6 +2609,20 @@ ReindexIndex(RangeVar *indexRelation, ReindexParams *params, bool isTopLevel)
persistence = get_rel_persistence(indOid);
relkind = get_rel_relkind(indOid);
+ /*
+ * If the index isn't partitioned, we can detect here if it has any oudated
+ * dependency.
+ */
+ if (relkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(indOid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indOid))));
+ return;
+ }
+
if (relkind == RELKIND_PARTITIONED_INDEX)
ReindexPartitions(indOid, params, isTopLevel);
else if ((params->options & REINDEXOPT_CONCURRENTLY) != 0 &&
@@ -3049,6 +3067,17 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel)
Assert(partkind == RELKIND_INDEX ||
partkind == RELKIND_RELATION);
+ /* Ignore indexes that don't depend on outdated dependency if needed */
+ if (partkind == RELKIND_INDEX &&
+ (params->options & REINDEXOPT_OUTDATED) != 0 &&
+ !index_has_outdated_dependency(partoid))
+ {
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(partoid))));
+ continue;
+ }
+
/* Save partition OID */
old_context = MemoryContextSwitchTo(reindex_context);
partitions = lappend_oid(partitions, partoid);
@@ -3307,7 +3336,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3389,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..94fc7b1c9e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,53 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ Assert(get_rel_relkind(indexOid) == RELKIND_INDEX ||
+ get_rel_relkind(indexOid) == RELKIND_PARTITIONED_INDEX);
+
+ /*
+ * Check for any oudated dependency.
+ */
+ if (index_has_outdated_dependency(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ if (((options & REINDEXOPT_OUTDATED) != 0))
+ ereport(NOTICE,
+ (errmsg("index \"%s\" has no outdated dependency",
+ get_rel_name(indexOid))));
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 4515401869..fab5dbc101 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -138,6 +138,7 @@ typedef struct IndexOrderByDistance
#define IndexScanIsValid(scan) PointerIsValid(scan)
extern Relation index_open(Oid relationId, LOCKMODE lockmode);
+extern Relation try_index_open(Oid relationId, LOCKMODE lockmode);
extern void index_close(Relation relation, LOCKMODE lockmode);
extern bool index_insert(Relation indexRelation,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index de70cb1212..001f091fb3 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2094,9 +2094,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
@@ -2107,9 +2109,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
objid
-------
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..959fad6b71 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,33 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes to reindex
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+NOTICE: table "reindex_coll_1" has no indexes that can be reindexed concurrently
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+NOTICE: index "reindex_coll_1_pkey" has no outdated dependency
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index dd5d208854..a6504512f7 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -836,9 +836,11 @@ WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE collate_test;
-REINDEX TABLE collate_part_0;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE collate_test;
+REINDEX (OUTDATED) TABLE collate_part_0;
REINDEX TABLE collate_part_1;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
@@ -847,9 +849,11 @@ UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
-REINDEX TABLE CONCURRENTLY collate_test;
+SET client_min_messages to WARNING; -- pg_toast index names aren't stable
+REINDEX (OUTDATED) TABLE CONCURRENTLY collate_test;
REINDEX TABLE CONCURRENTLY collate_part_0;
-REINDEX INDEX CONCURRENTLY icuidx17_part;
+REINDEX (OUTDATED) INDEX CONCURRENTLY icuidx17_part;
+RESET client_min_messages;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..4da6e2b5e8 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,24 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key) PARTITION BY LIST (id);
+CREATE TABLE reindex_coll_1 PARTITION OF reindex_coll FOR VALUES IN (1);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED) INDEX reindex_coll_1_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_pkey;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) TABLE reindex_coll_1;
+REINDEX (VERBOSE, OUTDATED, CONCURRENTLY) INDEX reindex_coll_1_pkey;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--rpt4sswdwkhtp4fw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9492a3c6b9..0eda6678ac 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26448,12 +26448,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26538,6 +26539,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index ff4dba8c36..aa66e6461f 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..0843f36580 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fcfd4..911c12ee2c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--mzh3msf6semlblkg
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v6-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command.
@ 2020-12-03 07:54 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Julien Rouhaud @ 2020-12-03 07:54 UTC (permalink / raw)
OUTDATED is added a new unreserved keyword.
When used, REINDEX will only process indexes that have an outdated dependency.
For now, only dependency on collations are supported but we'll likely support
other kind of dependency in the future.
Also add a new pg_index_has_outdated_dependency(regclass) SQL function, so
client code can filter such indexes if needed. This function will also be used
in a following commit to teach reindexdb to use this new OUTDATED option and
order the tables by the amount of work that will actually be done.
Catversion (should be) bumped.
Author: Julien Rouhaud <[email protected]>
Reviewed-by:
Discussion: https://postgr.es/m/20201203093143.GA64934%40nol
---
doc/src/sgml/func.sgml | 27 ++++--
doc/src/sgml/ref/reindex.sgml | 12 +++
src/backend/catalog/index.c | 107 ++++++++++++++++++++-
src/backend/commands/indexcmds.c | 12 ++-
src/backend/parser/gram.y | 4 +-
src/backend/utils/cache/relcache.c | 40 ++++++++
src/bin/psql/tab-complete.c | 2 +-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/include/parser/kwlist.h | 1 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/create_index.out | 10 ++
src/test/regress/sql/create_index.sql | 10 ++
13 files changed, 221 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index bf99f82149..2cf6e66234 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -26381,12 +26381,13 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
<para>
<xref linkend="functions-admin-index-table"/> shows the functions
- available for index maintenance tasks. (Note that these maintenance
- tasks are normally done automatically by autovacuum; use of these
- functions is only required in special cases.)
- These functions cannot be executed during recovery.
- Use of these functions is restricted to superusers and the owner
- of the given index.
+ available for index maintenance tasks. (Note that the maintenance
+ tasks performing actions on indexes are normally done automatically by
+ autovacuum; use of these functions is only required in special cases.)
+ The functions performing actions on indexes cannot be executed during
+ recovery.
+ Use of the functions performing actions on indexes is restricted to
+ superusers and the owner of the given index.
</para>
<table id="functions-admin-index-table">
@@ -26471,6 +26472,20 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
option.
</para></entry>
</row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_index_has_outdated_dependency</primary>
+ </indexterm>
+ <function>pg_index_has_outdated_dependency</function> ( <parameter>index</parameter> <type>regclass</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Check if the specified index has any outdated dependency. For now only
+ dependency on collations are supported.
+ </para></entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index b22d39eba9..2d94d49cde 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -26,6 +26,7 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<phrase>where <replaceable class="parameter">option</replaceable> can be one of:</phrase>
CONCURRENTLY [ <replaceable class="parameter">boolean</replaceable> ]
+ OUTDATED [ <replaceable class="parameter">boolean</replaceable> ]
TABLESPACE <replaceable class="parameter">new_tablespace</replaceable>
VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
</synopsis>
@@ -188,6 +189,17 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>OUTDATED</literal></term>
+ <listitem>
+ <para>
+ This option can be used to filter the list of indexes to rebuild and only
+ process indexes that have outdated dependencies. Fow now, the only
+ handle dependency is for the collation provider version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>TABLESPACE</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..571feac5db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -100,6 +100,12 @@ typedef struct
Oid pendingReindexedIndexes[FLEXIBLE_ARRAY_MEMBER];
} SerializedReindexState;
+typedef struct
+{
+ Oid relid; /* targetr index oid */
+ bool outdated; /* depends on at least on deprected collation? */
+} IndexHasOutdatedColl;
+
/* non-export function prototypes */
static bool relationHasPrimaryKey(Relation rel);
static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
@@ -1351,6 +1357,105 @@ index_check_collation_versions(Oid relid)
list_free(context.warned_colls);
}
+/*
+ * Detect if an index depends on at least one outdated collation.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_check_index_has_outdated_collation(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ IndexHasOutdatedColl *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Fast exit if we already found a outdated collation version. */
+ if (context->outdated)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId,
+ false);
+ if (!current_version)
+ return false;
+
+ if (!version || strcmp(version, current_version) != 0)
+ context->outdated = true;
+
+ return false;
+}
+
+Datum
+pg_index_has_outdated_dependency(PG_FUNCTION_ARGS)
+{
+ Oid indexOid = PG_GETARG_OID(0);
+ Relation rel;
+ bool isIndex;
+ bool res;
+
+ rel = try_relation_open(indexOid, AccessShareLock);
+
+ if (rel == NULL)
+ PG_RETURN_NULL();
+
+ isIndex = rel->rd_rel->relkind == RELKIND_INDEX;
+
+ if (!isIndex)
+ {
+ relation_close(rel, AccessShareLock);
+ PG_RETURN_NULL();
+ }
+
+ res = index_has_outdated_dependency(indexOid);
+
+ relation_close(rel, AccessShareLock);
+
+ PG_RETURN_BOOL(res);
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * collation version.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ */
+bool
+index_has_outdated_collation(Oid indexOid)
+{
+ ObjectAddress object;
+ IndexHasOutdatedColl context;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexOid;
+ object.objectSubId = 0;
+
+ context.relid = indexOid;
+ context.outdated = false;
+
+ visitDependenciesOf(&object, &do_check_index_has_outdated_collation,
+ &context);
+
+ return context.outdated;
+}
+
+/*
+ * Check whether the given index has a dependency with an outdated
+ * refobjversion.
+ * Caller must hold a suitable lock and make sure that the given Oid belongs to
+ * an index.
+ * For now, only dependency on collations are supported.
+ */
+bool
+index_has_outdated_dependency(Oid indexOid)
+{
+ return index_has_outdated_collation(indexOid);
+}
+
/*
* Update the version for collations. A callback for visitDependenciesOf().
*/
@@ -3991,7 +4096,7 @@ reindex_relation(Oid relid, int flags, ReindexParams *params)
* relcache to get this with a sequential scan if ignoring system
* indexes.)
*/
- indexIds = RelationGetIndexList(rel);
+ indexIds = RelationGetIndexListFiltered(rel, params->options);
if (flags & REINDEX_REL_SUPPRESS_INDEX_USE)
{
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 8bc652ecd3..0e1bcad101 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -2484,6 +2484,7 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
bool concurrently = false;
bool verbose = false;
char *tablespacename = NULL;
+ bool outdated_filter = false;
/* Parse option list */
foreach(lc, stmt->params)
@@ -2496,6 +2497,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
concurrently = defGetBoolean(opt);
else if (strcmp(opt->defname, "tablespace") == 0)
tablespacename = defGetString(opt);
+ else if (strcmp(opt->defname, "outdated") == 0)
+ outdated_filter = defGetBoolean(opt);
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -2510,7 +2513,8 @@ ExecReindex(ParseState *pstate, ReindexStmt *stmt, bool isTopLevel)
params.options =
(verbose ? REINDEXOPT_VERBOSE : 0) |
- (concurrently ? REINDEXOPT_CONCURRENTLY : 0);
+ (concurrently ? REINDEXOPT_CONCURRENTLY : 0) |
+ (outdated_filter ? REINDEXOPT_OUTDATED : 0);
/*
* Assign the tablespace OID to move indexes to, with InvalidOid to do
@@ -3307,7 +3311,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
RelationGetRelationName(heapRelation))));
/* Add all the valid indexes of relation to list */
- foreach(lc, RelationGetIndexList(heapRelation))
+ foreach(lc, RelationGetIndexListFiltered(heapRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc);
Relation indexRelation = index_open(cellOid,
@@ -3359,7 +3364,8 @@ ReindexRelationConcurrently(Oid relationOid, ReindexParams *params)
MemoryContextSwitchTo(oldcontext);
- foreach(lc2, RelationGetIndexList(toastRelation))
+ foreach(lc2, RelationGetIndexListFiltered(toastRelation,
+ params->options))
{
Oid cellOid = lfirst_oid(lc2);
Relation indexRelation = index_open(cellOid,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..a67cfa867c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -674,7 +674,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
NULLS_P NUMERIC
OBJECT_P OF OFF OFFSET OIDS OLD ON ONLY OPERATOR OPTION OPTIONS OR
- ORDER ORDINALITY OTHERS OUT_P OUTER_P
+ ORDER ORDINALITY OTHERS OUT_P OUTDATED OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
@@ -15430,6 +15430,7 @@ unreserved_keyword:
| OPTIONS
| ORDINALITY
| OTHERS
+ | OUTDATED
| OVER
| OVERRIDING
| OWNED
@@ -16002,6 +16003,7 @@ bare_label_keyword:
| ORDINALITY
| OTHERS
| OUT_P
+ | OUTDATED
| OUTER_P
| OVERLAY
| OVERRIDING
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..06545ee550 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4620,6 +4620,46 @@ RelationGetIndexList(Relation relation)
return result;
}
+/*
+ * RelationGetIndexListFiltered -- get a filtered list of indexes on this
+ * relation.
+ *
+ * Calls RelationGetIndexList and only keep indexes that have an outdated
+ * dependency. For now, only collation version dependency is supported.
+ */
+List *
+RelationGetIndexListFiltered(Relation relation, bits32 options)
+{
+ List *result,
+ *full_list;
+ ListCell *lc;
+
+ full_list = RelationGetIndexList(relation);
+
+ /* Fast exit if no filtering was asked, or if the list if empty. */
+ if (((options & REINDEXOPT_OUTDATED) == 0) || full_list == NIL)
+ return full_list;
+
+ result = NIL;
+ foreach(lc, full_list)
+ {
+ Oid indexOid = lfirst_oid(lc);
+
+ /*
+ * Check for outdated collation version dependency.
+ */
+ if (index_has_outdated_collation(indexOid))
+ {
+ result = lappend_oid(result, indexOid);
+ continue;
+ }
+
+ /* Didn't find any outdated dependency, index will be ignored. */
+ }
+
+ return result;
+}
+
/*
* RelationGetStatExtList
* get a list of OIDs of statistics objects on this relation
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208ac49..fac3e0476a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3674,7 +3674,7 @@ psql_completion(const char *text, int start, int end)
* one word, so the above test is correct.
*/
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
- COMPLETE_WITH("CONCURRENTLY", "TABLESPACE", "VERBOSE");
+ COMPLETE_WITH("CONCURRENTLY", "OUTDATED'", "TABLESPACE", "VERBOSE");
else if (TailMatches("TABLESPACE"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
}
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index e22d506436..298c0c633c 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -42,6 +42,7 @@ typedef struct ReindexParams
#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_OUTDATED 0x10/* outdated collation only */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -137,6 +138,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
bool *isnull);
extern void index_check_collation_versions(Oid relid);
+extern bool index_has_outdated_collation(Oid indexOid);
+extern bool index_has_outdated_dependency(Oid indexOid);
extern void index_update_collation_versions(Oid relid, Oid coll);
extern void index_build(Relation heapRelation,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3d3974f467..4874d33996 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -949,6 +949,10 @@
proname => 'pg_indexam_has_property', provolatile => 's',
prorettype => 'bool', proargtypes => 'oid text',
prosrc => 'pg_indexam_has_property' },
+{ oid => '8102', descr => 'test property of an index',
+ proname => 'pg_index_has_outdated_dependency', provolatile => 's',
+ prorettype => 'bool', proargtypes => 'regclass',
+ prosrc => 'pg_index_has_outdated_dependency' },
{ oid => '637', descr => 'test property of an index',
proname => 'pg_index_has_property', provolatile => 's', prorettype => 'bool',
proargtypes => 'regclass text', prosrc => 'pg_index_has_property' },
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..e6c725d9a6 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -295,6 +295,7 @@ PG_KEYWORD("order", ORDER, RESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("ordinality", ORDINALITY, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("others", OTHERS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("out", OUT_P, COL_NAME_KEYWORD, BARE_LABEL)
+PG_KEYWORD("outdated", OUTDATED, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("outer", OUTER_P, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("over", OVER, UNRESERVED_KEYWORD, AS_LABEL)
PG_KEYWORD("overlaps", OVERLAPS, TYPE_FUNC_NAME_KEYWORD, AS_LABEL)
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 2fcdf79323..a7a2272abd 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -45,6 +45,7 @@ extern void RelationClose(Relation relation);
*/
extern List *RelationGetFKeyList(Relation relation);
extern List *RelationGetIndexList(Relation relation);
+extern List *RelationGetIndexListFiltered(Relation relation, bits32 options);
extern List *RelationGetStatExtList(Relation relation);
extern Oid RelationGetPrimaryKeyIndex(Relation relation);
extern Oid RelationGetReplicaIndex(Relation relation);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..5f7c49c650 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2018,6 +2018,16 @@ INFO: index "reindex_verbose_pkey" was reindexed
\set VERBOSITY default
DROP TABLE reindex_verbose;
--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+NOTICE: table "reindex_coll" has no indexes to reindex
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+--
-- REINDEX CONCURRENTLY
--
CREATE TABLE concur_reindex_tab (c1 int);
diff --git a/src/test/regress/sql/create_index.sql b/src/test/regress/sql/create_index.sql
index 8bc76f7c6f..808cacee5d 100644
--- a/src/test/regress/sql/create_index.sql
+++ b/src/test/regress/sql/create_index.sql
@@ -790,6 +790,16 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
\set VERBOSITY default
DROP TABLE reindex_verbose;
+--
+-- REINDEX (OUTDATED)
+--
+CREATE TABLE reindex_coll(id integer primary key);
+\set VERBOSITY terse \\ -- suppress machine-dependent details
+-- no suitable index should be found
+REINDEX (OUTDATED) TABLE reindex_coll;
+\set VERBOSITY default
+DROP TABLE reindex_coll;
+
--
-- REINDEX CONCURRENTLY
--
--
2.30.1
--oao5se3im4osqowp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0002-Add-a-outdated-option-to-reindexdb.patch"
^ permalink raw reply [nested|flat] 76+ messages in thread
* Re: 001_password.pl fails with --without-readline
@ 2026-01-20 07:25 Soumya S Murali <[email protected]>
2026-01-30 20:13 ` Re: 001_password.pl fails with --without-readline Tom Lane <[email protected]>
0 siblings, 1 reply; 76+ messages in thread
From: Soumya S Murali @ 2026-01-20 07:25 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Oleg Tselebrovskiy <[email protected]>; pgsql-hackers
Hi all,
Thank you for the guidance and clarifications.
> =?UTF-8?B?T2xlZyBUc2VsZWJyb3Zza2l5?= <[email protected]> writes:
> >> I fetched all refs and tags locally and reset my tree to a clean
> >> origin/master, but I am unable to locate commit 34740b9 in my
> >> repository
>
> > Maybe your origin is out-of-date or is just different?
> > What does ‘git remote get-url origin’ say in your local repository?
>
> 34740b9 is there for me, but it's less than a day old:
>
> commit 34740b90bc123d645a3a71231b765b778bdcf049
> Author: Richard Guo <[email protected]>
> Date: Mon Jan 19 11:13:23 2026 +0900
>
> Fix unsafe pushdown of quals referencing grouping Vars
>
> However, none of these patches are touching any recently-modified
> code AFAIK, so they should apply cleanly even to a slightly out
> of date tree. I suspect Soumya's failure-to-apply problem has
> a different cause. The patches I posted were just "git diff"
> output, without a commit message, so I believe you can't use
> "git am" to apply them. Hoary old "patch -p1" ought to work
> though.
I missed the fact that the patches were raw git diff outputs rather
than meant for git am. I will retry applying them on a clean upstream
master tree and re-run the relevant test suites.
> =?UTF-8?B?T2xlZyBUc2VsZWJyb3Zza2l5?= <[email protected]> writes:
> >> While debugging that I got annoyed that a match failure results
> >> in a timeout exit with absolutely no data logged about what output
> >> the test got. So v3-0001 also changes timeout() --- which creates
> >> a timeout that aborts the test --- to timer() --- which does what
> >> the test author clearly expected, namely just stop waiting for
> >> more input. (There's a thread somewhere around here about making
> >> that change more globally, but I went ahead and did it here.)
>
> > I've found your thread about this - [1], and I agree, using
> > timer() is better here, we get the stdout and stderr of a timed-out
> > query
>
> Thanks for digging that up. After re-reading that thread I'm feeling
> nervous about changing timeout() to timer() in something we need to
> back-patch, so I'll leave that change out of the committed patch.
> We ought to raise the priority of making that happen, though.
>
> > Also, thanks for making both "pump until" blocks identical, it seemed
> > a little strange to have them be different.
>
> Yeah, I couldn't see a reason for that either.
I too agree that changing the behavior in a back-patch could be risky
and it makes sense to leave that part out for now while still
addressing the core test robustness issues. I will follow up once I
have re-applied the patches and completed testing.
Regards,
Soumya
^ permalink raw reply [nested|flat] 76+ messages in thread
* Re: 001_password.pl fails with --without-readline
2026-01-20 07:25 Re: 001_password.pl fails with --without-readline Soumya S Murali <[email protected]>
@ 2026-01-30 20:13 ` Tom Lane <[email protected]>
0 siblings, 0 replies; 76+ messages in thread
From: Tom Lane @ 2026-01-30 20:13 UTC (permalink / raw)
To: Soumya S Murali <[email protected]>; +Cc: Oleg Tselebrovskiy <[email protected]>; pgsql-hackers
Soumya S Murali <[email protected]> writes:
> I too agree that changing the behavior in a back-patch could be risky
> and it makes sense to leave that part out for now while still
> addressing the core test robustness issues. I will follow up once I
> have re-applied the patches and completed testing.
Since we're reaching the end of the January commitfest,
I went ahead and pushed these patches. There's still time to
revise them if your testing uncovers anything undesirable.
regards, tom lane
^ permalink raw reply [nested|flat] 76+ messages in thread
end of thread, other threads:[~2026-01-30 20:13 UTC | newest]
Thread overview: 76+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-12-05 20:07 [sqlsmith] Crash in gather_readnext Andreas Seltenreich <[email protected]>
2016-12-05 21:01 ` Robert Haas <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v7 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v6 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2020-12-03 07:54 [PATCH v5 1/2] Add a new OUTDATED filtering facility for REINDEX command. Julien Rouhaud <[email protected]>
2026-01-20 07:25 Re: 001_password.pl fails with --without-readline Soumya S Murali <[email protected]>
2026-01-30 20:13 ` Re: 001_password.pl fails with --without-readline Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox