public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Langote <[email protected]>
To: Tom Lane <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Jacob Champion <[email protected]>
Cc: David Rowley <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: generic plans and "initial" pruning
Date: Fri, 27 Jan 2023 16:01:05 +0900
Message-ID: <CA+HiwqEDr9m3NGrmiOgatCnRPwD95=MHgWQdwvnoMyQd3k9-Yw@mail.gmail.com> (raw)
In-Reply-To: <CA+HiwqGFm_5aUqPnt=WCarkJ2ZU6F8kD8pFeGurHP+NZWS8KQw@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<[email protected]>
<CA+HiwqFhQ8tLMLQAWYRWiBQUrWSLM8qhVzJ4B5jWr=orUXfSyA@mail.gmail.com>
<[email protected]>
<CA+HiwqGFm_5aUqPnt=WCarkJ2ZU6F8kD8pFeGurHP+NZWS8KQw@mail.gmail.com>
On Fri, Jan 20, 2023 at 12:52 PM Amit Langote <[email protected]> wrote:
> Alright, I'll try to get something out early next week. Thanks for
> all the pointers.
Sorry for the delay. Attached is what I've come up with so far.
I didn't actually go with calling the plancache on every lock taken on
a relation, that is, in ExecGetRangeTableRelation(). One thing about
doing it that way that I didn't quite like (or didn't see a clean
enough way to code) is the need to complicate the ExecInitNode()
traversal for handling the abrupt suspension of the ongoing setup of
the PlanState tree.
So, I decided to keep the current model of locking all the relations
that need to be locked before doing anything else in InitPlan(), much
as how AcquireExecutorLocks() does it. A new function called from
the top of InitPlan that I've called ExecLockRelationsIfNeeded() does
that locking after performing the initial pruning in the same manner
as the earlier patch did. That does mean that I needed to keep all
the adjustments of the pruning code that are required for such
out-of-ExecInitNode() invocation of initial pruning, including those
PartitionPruneResult to carry the result of that pruning for
ExecInitNode()-time reuse, though they no longer need be passed
through many unrelated interfaces.
Anyways, here's a description of the patches:
0001 adjusts various call sites of ExecutorStart() to cope with the
possibility of being asked to recreate a CachedPlan, if one is
involved. The main objective here is to have as little stuff as
sensible happen between GetCachedPlan() that returned the CachedPlan
and ExecutorStart() so as to minimize the chances of missing cleaning
up resources that must not be missed.
0002 is preparatory refactoring to make out-of-ExecInitNode()
invocation of pruning possible.
0003 moves the responsibility of CachedPlan validation locking into
ExecutorStart() as described above.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v31-0001-Move-ExecutorStart-closer-to-GetCachedPlan.patch (47.5K, 2-v31-0001-Move-ExecutorStart-closer-to-GetCachedPlan.patch)
download | inline diff:
From 4cfc3fdfb4c31a163fc3b0657be77927314cc1ca Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Fri, 20 Jan 2023 16:52:31 +0900
Subject: [PATCH v31 1/3] Move ExecutorStart() closer to GetCachedPlan()
This is in preparation for moving CachedPlan validation locking
into ExecutorStart(). The intent is to not have many steps between
GetCachedPlan() and ExecutorStart() so that if the latter invalidates
a CachedPlan, there's not much resource cleanup to worry about.
---
contrib/auto_explain/auto_explain.c | 9 +-
.../pg_stat_statements/pg_stat_statements.c | 8 +-
src/backend/commands/copyto.c | 5 +-
src/backend/commands/createas.c | 4 +-
src/backend/commands/explain.c | 148 ++++++---
src/backend/commands/extension.c | 3 +-
src/backend/commands/matview.c | 4 +-
src/backend/commands/portalcmds.c | 2 +-
src/backend/commands/prepare.c | 85 +++--
src/backend/executor/execMain.c | 18 +-
src/backend/executor/execParallel.c | 9 +-
src/backend/executor/functions.c | 3 +-
src/backend/executor/spi.c | 47 ++-
src/backend/nodes/Makefile | 1 +
src/backend/nodes/gen_node_support.pl | 2 +
src/backend/tcop/postgres.c | 12 +-
src/backend/tcop/pquery.c | 292 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 6 +
src/include/commands/explain.h | 8 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 6 +-
src/include/nodes/meson.build | 1 +
src/include/tcop/pquery.h | 3 +-
src/include/utils/portal.h | 2 +
24 files changed, 411 insertions(+), 271 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..0f20b97781 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,8 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags,
+ bool *replan);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -259,7 +260,7 @@ _PG_init(void)
* ExecutorStart hook: start up logging if needed
*/
static void
-explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
+explain_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan)
{
/*
* At the beginning of each top-level statement, decide whether we'll
@@ -296,9 +297,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ prev_ExecutorStart(queryDesc, eflags, replan);
else
- standard_ExecutorStart(queryDesc, eflags);
+ standard_ExecutorStart(queryDesc, eflags, replan);
if (auto_explain_enabled())
{
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index ad1fe44496..76348419ae 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -962,12 +962,12 @@ pgss_planner(Query *parse,
* ExecutorStart hook: start up tracking if needed
*/
static void
-pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
+pgss_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan)
{
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ prev_ExecutorStart(queryDesc, eflags, replan);
else
- standard_ExecutorStart(queryDesc, eflags);
+ standard_ExecutorStart(queryDesc, eflags, replan);
/*
* If query has queryId zero, don't track it. This prevents double
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 8043b4e9b1..b6d8fa59d5 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -568,7 +569,7 @@ BeginCopyTo(ParseState *pstate,
*
* ExecutorStart computes a result tupdesc for us
*/
- ExecutorStart(cstate->queryDesc, 0);
+ ExecutorStart(cstate->queryDesc, 0, NULL);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index d6c6d514f3..ee33f02602 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,12 +325,12 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
/* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ ExecutorStart(queryDesc, GetIntoRelEFlags(into), NULL);
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0L, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5212a64b1e..fcb227533c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -406,12 +407,94 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv, NULL);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
+
+ /* One pushed by ExplainQueryDesc(). */
+ PopActiveSnapshot();
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * On return, *replan is set to true if cplan is found to have been
+ * invalidated since its creation.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv,
+ bool *replan)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ if (replan)
+ *replan = false;
+ ExecutorStart(queryDesc, eflags, replan);
+ if (replan && *replan)
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -515,30 +598,18 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
+ PlannedStmt *plannedstmt = queryDesc->plannedstmt;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
-
/*
* We always collect timing for the entire statement, even when node-level
* timing is off, so we don't look at es->timing here. (We could skip
@@ -546,38 +617,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -658,8 +697,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
FreeQueryDesc(queryDesc);
- PopActiveSnapshot();
-
/* We need a CCI just in case query expanded to multiple plans */
if (es->analyze)
CommandCounterIncrement();
@@ -4854,6 +4891,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b1509cc505..1493b99beb 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -780,11 +780,12 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ ExecutorStart(qdesc, 0, NULL);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index fb30d2595c..e13b344ba3 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -409,12 +409,12 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ ExecutorStart(queryDesc, 0, NULL);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0L, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 8a3cf98cce..9fd27bf07a 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -143,7 +143,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
+ PortalStart(portal, params, 0, GetActiveSnapshot(), NULL);
Assert(portal->strategy == PORTAL_ONE_SELECT);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..c1fa1b72be 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -155,6 +155,7 @@ ExecuteQuery(ParseState *pstate,
PreparedStatement *entry;
CachedPlan *cplan;
List *plan_list;
+ bool replan;
ParamListInfo paramLI = NULL;
EState *estate = NULL;
Portal portal;
@@ -193,6 +194,7 @@ ExecuteQuery(ParseState *pstate,
entry->plansource->query_string);
/* Replan if needed, and increment plan refcount for portal */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI, NULL, NULL);
plan_list = cplan->stmt_list;
@@ -251,9 +253,16 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan,
+ * it must be recreated if *replan is set.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ PortalStart(portal, paramLI, eflags, GetActiveSnapshot(), &replan);
+
+ if (replan)
+ {
+ MarkPortalFailed(portal);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +583,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -583,6 +592,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
instr_time planduration;
BufferUsage bufusage_start,
bufusage;
+ bool replan = true;
if (es->buffers)
bufusage_start = pgBufferUsage;
@@ -618,38 +628,57 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
- cplan = GetCachedPlan(entry->plansource, paramLI,
- CurrentResourceOwner, queryEnv);
+ while (replan)
+ {
+ cplan = GetCachedPlan(entry->plansource, paramLI,
+ CurrentResourceOwner, queryEnv);
- INSTR_TIME_SET_CURRENT(planduration);
- INSTR_TIME_SUBTRACT(planduration, planstart);
+ INSTR_TIME_SET_CURRENT(planduration);
+ INSTR_TIME_SUBTRACT(planduration, planstart);
- /* calc differences of buffer counters. */
- if (es->buffers)
- {
- memset(&bufusage, 0, sizeof(BufferUsage));
- BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
- }
+ /* calc differences of buffer counters. */
+ if (es->buffers)
+ {
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
+ }
- plan_list = cplan->stmt_list;
+ plan_list = cplan->stmt_list;
- /* Explain each query */
- foreach(p, plan_list)
- {
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
+ /* Explain each query */
+ foreach(p, plan_list)
+ {
+ PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
- if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
- else
- ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
- paramLI, queryEnv);
+ if (pstmt->commandType != CMD_UTILITY)
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv,
+ &replan);
+ if (replan)
+ {
+ ExplainResetOutput(es);
+ break;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+
+ /* One pushed by ExplainQueryDesc(). */
+ PopActiveSnapshot();
+ }
+ else
+ ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
+ paramLI, queryEnv);
- /* No need for CommandCounterIncrement, as ExplainOnePlan did it */
+ /* No need for CommandCounterIncrement, as ExplainOnePlan did it */
- /* Separate plans with an appropriate separator */
- if (lnext(plan_list, p) != NULL)
- ExplainSeparatePlans(es);
+ /* Separate plans with an appropriate separator */
+ if (lnext(plan_list, p) != NULL)
+ ExplainSeparatePlans(es);
+ }
}
if (estate)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a5115b9c1f..45c999bcdb 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -119,6 +119,11 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * replan must be non-NULL when executing a cached query plan. On return,
+ * *replan is set if queryDesc->cplan is found to have been invalidated. In
+ * that case, callers must recreate the CachedPlan before retrying the
+ * execution.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -129,8 +134,10 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
* ----------------------------------------------------------------
*/
void
-ExecutorStart(QueryDesc *queryDesc, int eflags)
+ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan)
{
+ Assert(replan != NULL || queryDesc->cplan == NULL);
+
/*
* In some cases (e.g. an EXECUTE statement) a query execution will skip
* parse analysis, which means that the query_id won't be reported. Note
@@ -140,13 +147,13 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
+ (*ExecutorStart_hook) (queryDesc, eflags, replan);
else
- standard_ExecutorStart(queryDesc, eflags);
+ standard_ExecutorStart(queryDesc, eflags, replan);
}
void
-standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
+standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan)
{
EState *estate;
MemoryContext oldcontext;
@@ -2797,7 +2804,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index aa3f283453..5f97f5353f 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the containing plan tree may have come from one in the
+ * leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1431,7 +1436,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ ExecutorStart(queryDesc, fpes->eflags, NULL);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 50e06ec693..df37bfb4ed 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -867,7 +868,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+ ExecutorStart(es->qd, eflags, NULL);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 61f03e3999..9a3398b591 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1578,6 +1578,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
CachedPlanSource *plansource;
CachedPlan *cplan;
List *stmt_list;
+ bool replan;
char *query_string;
Snapshot snapshot;
MemoryContext oldcontext;
@@ -1657,6 +1658,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
*/
/* Replan if needed, and increment plan refcount for portal */
+replan:
cplan = GetCachedPlan(plansource, paramLI, NULL, _SPI_current->queryEnv);
stmt_list = cplan->stmt_list;
@@ -1766,9 +1768,16 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if *replan is set.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ PortalStart(portal, paramLI, 0, snapshot, &replan);
+
+ if (replan)
+ {
+ MarkPortalFailed(portal);
+ goto replan;
+ }
Assert(portal->strategy != PORTAL_MULTI_QUERY);
@@ -2548,6 +2557,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2657,6 +2667,8 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
+ bool replan = false;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2664,14 +2676,28 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+ ExecutorStart(qdesc, eflags, &replan);
+ if (replan)
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2846,10 +2872,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2893,14 +2918,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile
index af12c64878..7fb0d2d202 100644
--- a/src/backend/nodes/Makefile
+++ b/src/backend/nodes/Makefile
@@ -52,6 +52,7 @@ node_headers = \
access/tsmapi.h \
commands/event_trigger.h \
commands/trigger.h \
+ executor/execdesc.h \
executor/tuptable.h \
foreign/fdwapi.h \
nodes/bitmapset.h \
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index b3c1ead496..74f83f12a6 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -63,6 +63,7 @@ my @all_input_files = qw(
access/tsmapi.h
commands/event_trigger.h
commands/trigger.h
+ executor/execdesc.h
executor/tuptable.h
foreign/fdwapi.h
nodes/bitmapset.h
@@ -87,6 +88,7 @@ my @nodetag_only_files = qw(
access/tsmapi.h
commands/event_trigger.h
commands/trigger.h
+ executor/execdesc.h
executor/tuptable.h
foreign/fdwapi.h
nodes/lockoptions.h
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 470b734e9e..1617b93ecc 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1195,7 +1195,7 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ PortalStart(portal, NULL, 0, InvalidSnapshot, NULL);
/*
* Select the appropriate output format: text unless we are doing a
@@ -1597,6 +1597,7 @@ exec_bind_message(StringInfo input_message)
int16 *rformats = NULL;
CachedPlanSource *psrc;
CachedPlan *cplan;
+ bool replan;
Portal portal;
char *query_string;
char *saved_stmt_name;
@@ -1971,6 +1972,7 @@ exec_bind_message(StringInfo input_message)
* will be generated in MessageContext. The plan refcount will be
* assigned to the Portal, so it will be released at portal destruction.
*/
+replan:
cplan = GetCachedPlan(psrc, params, NULL, NULL);
/*
@@ -1993,7 +1995,13 @@ exec_bind_message(StringInfo input_message)
/*
* And we're ready to start portal execution.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ PortalStart(portal, params, 0, InvalidSnapshot, &replan);
+
+ if (replan)
+ {
+ MarkPortalFailed(portal);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5f0248acc5..97de5c53e3 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -75,8 +71,10 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
{
QueryDesc *qd = (QueryDesc *) palloc(sizeof(QueryDesc));
+ qd->type = T_QueryDesc;
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +114,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0L, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -427,15 +345,16 @@ FetchStatementTargetList(Node *stmt)
* to be used for cursors).
*
* On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * tupdesc (if any) is known, unless *replan is set to true, in which case,
+ * the caller must retry after generating a new CachedPlan.
*/
void
PortalStart(Portal portal, ParamListInfo params,
- int eflags, Snapshot snapshot)
+ int eflags, Snapshot snapshot,
+ bool *replan)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
int myeflags;
@@ -443,20 +362,21 @@ PortalStart(Portal portal, ParamListInfo params,
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
+ if (replan)
+ *replan = false;
+
/*
* Set up global portal context pointers.
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +392,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +415,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +424,48 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
- ExecutorStart(queryDesc, myeflags);
+ ExecutorStart(queryDesc, myeflags, replan);
+ if (replan && *replan)
+ {
+ Assert(queryDesc->cplan);
+ PopActiveSnapshot();
+ goto early_exit;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +499,61 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool pushed_active_snapshot = false;
+
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /* Must set snapshot before starting executor. */
+ if (!pushed_active_snapshot && !is_utility)
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ pushed_active_snapshot = true;
+ }
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ pushed_active_snapshot ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalMultiRun() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ if (!is_utility)
+ {
+ ExecutorStart(queryDesc, 0, replan);
+ if (replan && *replan)
+ {
+ Assert(queryDesc->cplan);
+ if (pushed_active_snapshot)
+ PopActiveSnapshot();
+ goto early_exit;
+ }
+ }
+ }
+
+ if (pushed_active_snapshot)
+ PopActiveSnapshot();
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,19 +566,18 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+early_exit:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
-
- portal->status = PORTAL_READY;
}
/*
@@ -1193,7 +1164,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1185,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = lfirst_node(QueryDesc, qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1241,7 +1213,7 @@ PortalRunMulti(Portal portal,
*/
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
/* If told to, register the snapshot and save in portal */
if (setHoldSnapshot)
@@ -1271,23 +1243,38 @@ PortalRunMulti(Portal portal,
else
UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0L, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1346,8 +1333,19 @@ PortalRunMulti(Portal portal,
* Increment command counter between queries, but not after the last
* one.
*/
- if (lnext(portal->stmts, stmtlist_item) != NULL)
+ if (lnext(portal->qdescs, qdesc_item) != NULL)
CommandCounterIncrement();
+
+ /* portal->queryDesc is free'd by PortalCleanup(). */
+ if (qdesc != portal->queryDesc)
+ {
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
+ }
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..3ad80c7ecb 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /* initialize portal's query context to store QueryDescs */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..ea35adfb3d 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -87,7 +87,12 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv,
+ bool *replan);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -103,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index e7e25c057e..63f3d09804 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -62,7 +62,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags, bool *replan);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -187,8 +187,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern void ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan);
+extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build
index efe0834afb..a8fdd9e176 100644
--- a/src/include/nodes/meson.build
+++ b/src/include/nodes/meson.build
@@ -13,6 +13,7 @@ node_support_input_i = [
'access/tsmapi.h',
'commands/event_trigger.h',
'commands/trigger.h',
+ 'executor/execdesc.h',
'executor/tuptable.h',
'foreign/fdwapi.h',
'nodes/bitmapset.h',
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..08783f1b43 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -30,7 +30,8 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
extern void PortalStart(Portal portal, ParamListInfo params,
- int eflags, Snapshot snapshot);
+ int eflags, Snapshot snapshot,
+ bool *replan);
extern void PortalSetResultFormat(Portal portal, int nFormats,
int16 *formats);
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v31-0003-Move-CachedPlan-validation-locking-to-ExecutorSt.patch (38.4K, 3-v31-0003-Move-CachedPlan-validation-locking-to-ExecutorSt.patch)
download | inline diff:
From 0522447f5816211ac3e32ebc6920d7f7805718d6 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Thu, 26 Jan 2023 10:52:24 +0900
Subject: [PATCH v31 3/3] Move CachedPlan validation locking to ExecutorStart()
---
src/backend/executor/execMain.c | 163 +++++++++++++++++++++++--
src/backend/executor/execParallel.c | 38 +++++-
src/backend/executor/execPartition.c | 90 +++++++++++---
src/backend/executor/execUtils.c | 8 +-
src/backend/executor/nodeAppend.c | 11 +-
src/backend/executor/nodeMergeAppend.c | 5 +-
src/backend/optimizer/plan/setrefs.c | 36 ++++++
src/backend/utils/cache/plancache.c | 146 +++++++---------------
src/include/executor/execPartition.h | 8 +-
src/include/executor/execdesc.h | 6 +
src/include/executor/executor.h | 2 +
src/include/nodes/execnodes.h | 1 +
src/include/nodes/pathnodes.h | 4 +-
src/include/nodes/plannodes.h | 31 ++++-
src/include/utils/plancache.h | 1 +
15 files changed, 404 insertions(+), 146 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 45c999bcdb..68743d5f66 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -49,6 +49,7 @@
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/execdebug.h"
+#include "executor/execPartition.h"
#include "executor/nodeSubplan.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
@@ -64,6 +65,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/plancache.h"
#include "utils/rls.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -79,7 +81,12 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static void InitPlan(QueryDesc *queryDesc, int eflags, bool *replan);
+static void ExecLockRelationsIfNeeded(QueryDesc *queryDesc, bool *replan);
+static Bitmapset *ExecDoInitialPartitionPruning(PlannedStmt *stmt,
+ EState *estate);
+static void AcquireExecutorLocks(Bitmapset *lockRelids, EState *estate,
+ bool acquire);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -270,7 +277,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ InitPlan(queryDesc, eflags, replan);
MemoryContextSwitchTo(oldcontext);
}
@@ -801,7 +808,7 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
* ----------------------------------------------------------------
*/
static void
-InitPlan(QueryDesc *queryDesc, int eflags)
+InitPlan(QueryDesc *queryDesc, int eflags, bool *replan)
{
CmdType operation = queryDesc->operation;
PlannedStmt *plannedstmt = queryDesc->plannedstmt;
@@ -814,19 +821,26 @@ InitPlan(QueryDesc *queryDesc, int eflags)
int i;
/*
- * Do permissions checks and save the list for later use.
- */
- ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
- estate->es_rteperminfos = plannedstmt->permInfos;
-
- /*
- * initialize the node's execution state
+ * Initialize es_range_table and es_relations.
*/
ExecInitRangeTable(estate, rangeTable);
estate->es_plannedstmt = plannedstmt;
estate->es_part_prune_infos = plannedstmt->partPruneInfos;
+ /*
+ * Acquire locks on relations referenced in the plan if it comes
+ * from a CachedPlan after performing "initial" partition pruning.
+ * Results of pruning, if any, are saved in es_part_prune_results.
+ */
+ ExecLockRelationsIfNeeded(queryDesc, replan);
+
+ /*
+ * Do permissions checks and save the list for later use.
+ */
+ ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
+ estate->es_rteperminfos = plannedstmt->permInfos;
+
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
*/
@@ -982,6 +996,133 @@ InitPlan(QueryDesc *queryDesc, int eflags)
queryDesc->planstate = planstate;
}
+/*
+ * ExecLockRelationsIfNeeded
+ * Lock relations that a query's plan depends on if the plan comes
+ * from a CachedPlan
+ *
+ * On return, we have all acquired the locks needed to run the plan.
+ * Also *replan is set to true if acquiring those locks would have invalidated
+ * the CachedPlan.
+ */
+static void
+ExecLockRelationsIfNeeded(QueryDesc *queryDesc, bool *replan)
+{
+ PlannedStmt *plannedstmt = queryDesc->plannedstmt;
+ EState *estate = queryDesc->estate;
+ CachedPlan *cplan = queryDesc->cplan;
+ Bitmapset *allLockRelids;
+
+ /* Nothing to do if the plan tree is not cached. */
+ if (cplan == NULL || cplan->is_oneshot)
+ return;
+
+ Assert(plannedstmt);
+ Assert(replan);
+ *replan = false;
+
+ /*
+ * Temporarily signal to ExecGetRangeTableRelation() that it must take
+ * take a lock. This is needed for CreatePartitionPruneState() to be
+ * able to open parent partitioned tables using
+ * ExecGetRangeTableRelation().
+ */
+ estate->es_top_eflags |= EXEC_FLAG_GET_LOCKS;
+
+ allLockRelids = plannedstmt->minLockRelids;
+ if (plannedstmt->containsInitialPruning)
+ {
+ Bitmapset *partRelids = ExecDoInitialPartitionPruning(plannedstmt,
+ estate);
+
+ allLockRelids = bms_add_members(allLockRelids, partRelids);
+ }
+
+ /* Done with it. */
+ estate->es_top_eflags &= ~EXEC_FLAG_GET_LOCKS;
+
+ /* Acquire locks. */
+ AcquireExecutorLocks(allLockRelids, estate, true);
+
+ /* Check if acquiring those locks has invalidated the plan. */
+ *replan = !CachedPlanStillValid(cplan);
+
+ /* Release useless locks if needed. */
+ if (*replan)
+ AcquireExecutorLocks(allLockRelids, estate, false);
+}
+
+/*
+ * ExecDoInitialPartitionPruning
+ * Perform initial partition pruning if needed by the plan
+ *
+ * The return value is the set of RT indexes of surviving partitions.
+ * A list of PartitionPruneResult with an element for each in
+ * plannedstmt->partPruneInfos is saved in estate->es_part_prune_results.
+ */
+static Bitmapset *
+ExecDoInitialPartitionPruning(PlannedStmt *plannedstmt, EState *estate)
+{
+ ListCell *lc;
+ Bitmapset *lockPartRelids = NULL;
+
+ Assert(plannedstmt->containsInitialPruning);
+ Assert(plannedstmt->partPruneInfos);
+
+ foreach(lc, plannedstmt->partPruneInfos)
+ {
+ PartitionPruneInfo *pruneinfo = lfirst_node(PartitionPruneInfo, lc);
+ PartitionPruneState *prunestate;
+ PartitionPruneResult *pruneresult;
+ Bitmapset *validsubplans;
+
+ /* No PlanState here; unnecessary for "initial" pruning. */
+ prunestate = ExecCreatePartitionPruneState(NULL, estate, pruneinfo,
+ true, false);
+ validsubplans = ExecFindMatchingSubPlans(prunestate, true,
+ &lockPartRelids);
+
+ pruneresult = makeNode(PartitionPruneResult);
+ pruneresult->root_parent_relids = bms_copy(pruneinfo->root_parent_relids);
+ pruneresult->validsubplans = bms_copy(validsubplans);
+ estate->es_part_prune_results = lappend(estate->es_part_prune_results,
+ pruneresult);
+ }
+
+ return lockPartRelids;
+}
+
+/*
+ * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
+ * or release them if acquire is false.
+ */
+static void
+AcquireExecutorLocks(Bitmapset *lockRelids, EState *estate, bool acquire)
+{
+ int rti;
+
+ rti = -1;
+ while ((rti = bms_next_member(lockRelids, rti)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(rti, estate);
+
+ if (!(rte->rtekind == RTE_RELATION ||
+ (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
+ continue;
+
+ /*
+ * Acquire the appropriate type of lock on each relation OID. Note
+ * that we don't actually try to open the rel, and hence will not
+ * fail if it's been dropped entirely --- we'll just transiently
+ * acquire a non-conflicting lock.
+ */
+ if (acquire)
+ LockRelationOid(rte->relid, rte->rellockmode);
+ else
+ UnlockRelationOid(rte->relid, rte->rellockmode);
+ }
+}
+
/*
* Check that a proposed result relation is a legal target for the operation
*
@@ -1396,7 +1537,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner.
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 1f5d6d4d64..5c967451ce 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -66,6 +66,7 @@
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xE000000000000008)
#define PARALLEL_KEY_JIT_INSTRUMENTATION UINT64CONST(0xE000000000000009)
#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xE00000000000000A)
+#define PARALLEL_KEY_PARTITION_PRUNE_RESULTS UINT64CONST(0xE00000000000000B)
#define PARALLEL_TUPLE_QUEUE_SIZE 65536
@@ -599,12 +600,15 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
FixedParallelExecutorState *fpes;
char *pstmt_data;
char *pstmt_space;
+ char *part_prune_results_data;
+ char *part_prune_results_space;
char *paramlistinfo_space;
BufferUsage *bufusage_space;
WalUsage *walusage_space;
SharedExecutorInstrumentation *instrumentation = NULL;
SharedJitInstrumentation *jit_instrumentation = NULL;
int pstmt_len;
+ int part_prune_results_len;
int paramlistinfo_len;
int instrumentation_len = 0;
int jit_instrumentation_len = 0;
@@ -633,6 +637,7 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
/* Fix up and serialize plan to be sent to workers. */
pstmt_data = ExecSerializePlan(planstate->plan, estate);
+ part_prune_results_data = nodeToString(estate->es_part_prune_results);
/* Create a parallel context. */
pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers);
@@ -659,6 +664,11 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
shm_toc_estimate_chunk(&pcxt->estimator, pstmt_len);
shm_toc_estimate_keys(&pcxt->estimator, 1);
+ /* Estimate space for serialized List of PartitionPruneResult. */
+ part_prune_results_len = strlen(part_prune_results_data) + 1;
+ shm_toc_estimate_chunk(&pcxt->estimator, part_prune_results_len);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+
/* Estimate space for serialized ParamListInfo. */
paramlistinfo_len = EstimateParamListSpace(estate->es_param_list_info);
shm_toc_estimate_chunk(&pcxt->estimator, paramlistinfo_len);
@@ -753,6 +763,12 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
memcpy(pstmt_space, pstmt_data, pstmt_len);
shm_toc_insert(pcxt->toc, PARALLEL_KEY_PLANNEDSTMT, pstmt_space);
+ /* Store serialized List of PartitionPruneResult */
+ part_prune_results_space = shm_toc_allocate(pcxt->toc, part_prune_results_len);
+ memcpy(part_prune_results_space, part_prune_results_data, part_prune_results_len);
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARTITION_PRUNE_RESULTS,
+ part_prune_results_space);
+
/* Store serialized ParamListInfo. */
paramlistinfo_space = shm_toc_allocate(pcxt->toc, paramlistinfo_len);
shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARAMLISTINFO, paramlistinfo_space);
@@ -1234,8 +1250,11 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
int instrument_options)
{
char *pstmtspace;
+ char *part_prune_results_space;
char *paramspace;
PlannedStmt *pstmt;
+ QueryDesc *queryDesc;
+ List *part_prune_results;
ParamListInfo paramLI;
char *queryString;
@@ -1246,6 +1265,11 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
pstmtspace = shm_toc_lookup(toc, PARALLEL_KEY_PLANNEDSTMT, false);
pstmt = (PlannedStmt *) stringToNode(pstmtspace);
+ /* Reconstruct leader-supplied PartitionPruneResult. */
+ part_prune_results_space =
+ shm_toc_lookup(toc, PARALLEL_KEY_PARTITION_PRUNE_RESULTS, false);
+ part_prune_results = (List *) stringToNode(part_prune_results_space);
+
/* Reconstruct ParamListInfo. */
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
@@ -1255,11 +1279,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
* here even if the containing plan tree may have come from one in the
* leader.
*/
- return CreateQueryDesc(pstmt,
- NULL,
- queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- receiver, paramLI, NULL, instrument_options);
+ queryDesc = CreateQueryDesc(pstmt,
+ NULL,
+ queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ receiver, paramLI, NULL, instrument_options);
+
+ queryDesc->part_prune_results = part_prune_results;
+
+ return queryDesc;
}
/*
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 4b91bb7403..09e0d7aa9c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -196,7 +196,8 @@ static void PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate,
static void find_matching_subplans_recurse(PartitionPruningData *prunedata,
PartitionedRelPruningData *pprune,
bool initial_prune,
- Bitmapset **validsubplans);
+ Bitmapset **validsubplans,
+ Bitmapset **scan_leafpart_rtis);
/*
@@ -1782,8 +1783,10 @@ adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap)
*
* On return, *initially_valid_subplans is assigned the set of indexes of
* child subplans that must be initialized along with the parent plan node.
- * Initial pruning is performed here if needed and in that case only the
- * surviving subplans' indexes are added.
+ * That set is computed by either performing the "initial pruning" here or
+ * reusing the one present in EState.es_part_prune_results[part_prune_index]
+ * if it has been set, which it would be if ExecDoInitialPartitionPruning()
+ * would have done the initial pruning.
*
* If subplans are indeed pruned, subplan_map arrays contained in the returned
* PartitionPruneState are re-sequenced to not count those, though only if the
@@ -1796,9 +1799,10 @@ ExecInitPartitionPruning(PlanState *planstate,
Bitmapset *root_parent_relids,
Bitmapset **initially_valid_subplans)
{
- PartitionPruneState *prunestate;
+ PartitionPruneState *prunestate = NULL;
EState *estate = planstate->state;
PartitionPruneInfo *pruneinfo;
+ PartitionPruneResult *pruneresult = NULL;
/* Obtain the pruneinfo we need, and make sure it's the right one */
pruneinfo = list_nth(estate->es_part_prune_infos, part_prune_index);
@@ -1814,22 +1818,56 @@ ExecInitPartitionPruning(PlanState *planstate,
/* We may need an expression context to evaluate partition exprs */
ExecAssignExprContext(estate, planstate);
- /* Create the working data structure for pruning */
- prunestate = ExecCreatePartitionPruneState(planstate, estate, pruneinfo,
- pruneinfo->needs_init_pruning,
- pruneinfo->needs_exec_pruning);
+ /* Initial pruning already done if es_part_prune_results has been set. */
+ if (estate->es_part_prune_results)
+ {
+ pruneresult = list_nth_node(PartitionPruneResult,
+ estate->es_part_prune_results,
+ part_prune_index);
+ if (!bms_equal(root_parent_relids, pruneinfo->root_parent_relids))
+ ereport(ERROR,
+ errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg_internal("mismatching PartitionPruneInfo and PartitionPruneResult at part_prune_index %d",
+ part_prune_index),
+ errdetail_internal("prunresult relids %s, pruneinfo relids %s",
+ bmsToString(pruneresult->root_parent_relids),
+ bmsToString(pruneinfo->root_parent_relids)));
+ }
+
+ if (pruneresult == NULL || pruneinfo->needs_exec_pruning)
+ {
+ /* We may need an expression context to evaluate partition exprs */
+ ExecAssignExprContext(estate, planstate);
+
+ /*
+ * Create the working data structure for pruning. No need to consider
+ * initial pruning steps if we have a PartitionPruneResult.
+ */
+ prunestate = ExecCreatePartitionPruneState(planstate, estate,
+ pruneinfo,
+ pruneresult == NULL,
+ pruneinfo->needs_exec_pruning);
+ }
/*
* Perform an initial partition prune pass, if required.
*/
- if (prunestate->do_initial_prune)
- *initially_valid_subplans = ExecFindMatchingSubPlans(prunestate, true);
+ if (pruneresult)
+ {
+ *initially_valid_subplans = bms_copy(pruneresult->validsubplans);
+ }
+ else if (prunestate && prunestate->do_initial_prune)
+ {
+ *initially_valid_subplans = ExecFindMatchingSubPlans(prunestate, true,
+ NULL);
+ }
else
{
- /* No pruning, so we'll need to initialize all subplans */
+ /* No initial pruning, so we'll need to initialize all subplans */
Assert(n_total_subplans > 0);
*initially_valid_subplans = bms_add_range(NULL, 0,
n_total_subplans - 1);
+ return prunestate;
}
/*
@@ -1837,7 +1875,8 @@ ExecInitPartitionPruning(PlanState *planstate,
* that were removed above due to initial pruning. No need to do this if
* no steps were removed.
*/
- if (bms_num_members(*initially_valid_subplans) < n_total_subplans)
+ if (prunestate &&
+ bms_num_members(*initially_valid_subplans) < n_total_subplans)
{
/*
* We can safely skip this when !do_exec_prune, even though that
@@ -2295,10 +2334,14 @@ PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate,
* Pass initial_prune if PARAM_EXEC Params cannot yet be evaluated. This
* differentiates the initial executor-time pruning step from later
* runtime pruning.
+ *
+ * RT indexes of leaf partitions scanned by the chosen subplans are added to
+ * *scan_leafpart_rtis if the pointer is non-NULL.
*/
Bitmapset *
ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
- bool initial_prune)
+ bool initial_prune,
+ Bitmapset **scan_leafpart_rtis)
{
Bitmapset *result = NULL;
MemoryContext oldcontext;
@@ -2333,10 +2376,10 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
*/
pprune = &prunedata->partrelprunedata[0];
find_matching_subplans_recurse(prunedata, pprune, initial_prune,
- &result);
+ &result, scan_leafpart_rtis);
/* Expression eval may have used space in ExprContext too */
- if (pprune->exec_pruning_steps)
+ if (pprune->exec_pruning_steps && !initial_prune)
ResetExprContext(pprune->exec_context.exprcontext);
}
@@ -2347,6 +2390,8 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
/* Copy result out of the temp context before we reset it */
result = bms_copy(result);
+ if (scan_leafpart_rtis)
+ *scan_leafpart_rtis = bms_copy(*scan_leafpart_rtis);
MemoryContextReset(prunestate->prune_context);
@@ -2357,13 +2402,15 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
* find_matching_subplans_recurse
* Recursive worker function for ExecFindMatchingSubPlans
*
- * Adds valid (non-prunable) subplan IDs to *validsubplans
+ * Adds valid (non-prunable) subplan IDs to *validsubplans and RT indexes of
+ * of the corresponding leaf partitions to *scan_leafpart_rtis (if asked for).
*/
static void
find_matching_subplans_recurse(PartitionPruningData *prunedata,
PartitionedRelPruningData *pprune,
bool initial_prune,
- Bitmapset **validsubplans)
+ Bitmapset **validsubplans,
+ Bitmapset **scan_leafpart_rtis)
{
Bitmapset *partset;
int i;
@@ -2390,8 +2437,14 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata,
while ((i = bms_next_member(partset, i)) >= 0)
{
if (pprune->subplan_map[i] >= 0)
+ {
*validsubplans = bms_add_member(*validsubplans,
pprune->subplan_map[i]);
+ Assert(pprune->rti_map[i] > 0);
+ if (scan_leafpart_rtis)
+ *scan_leafpart_rtis = bms_add_member(*scan_leafpart_rtis,
+ pprune->rti_map[i]);
+ }
else
{
int partidx = pprune->subpart_map[i];
@@ -2399,7 +2452,8 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata,
if (partidx >= 0)
find_matching_subplans_recurse(prunedata,
&prunedata->partrelprunedata[partidx],
- initial_prune, validsubplans);
+ initial_prune, validsubplans,
+ scan_leafpart_rtis);
else
{
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c33a3c0bec..035ed8a872 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -140,6 +140,7 @@ CreateExecutorState(void)
estate->es_param_exec_vals = NULL;
estate->es_queryEnv = NULL;
+ estate->es_part_prune_results = NIL;
estate->es_query_cxt = qcontext;
@@ -800,7 +801,12 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ /*
+ * Must take a lock on the relation if we got here by way of
+ * ExecLockRelationsIfNeeded().
+ */
+ if (!IsParallelWorker() &&
+ (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0)
{
/*
* In a normal query, we should already have the appropriate lock,
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index cb25499b3f..2f585793da 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -156,7 +156,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
* subplan, we can fill as_valid_subplans immediately, preventing
* later calls to ExecFindMatchingSubPlans.
*/
- if (!prunestate->do_exec_prune && nplans > 0)
+ if (appendstate->as_prune_state == NULL ||
+ (!appendstate->as_prune_state->do_exec_prune && nplans > 0))
appendstate->as_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
}
else
@@ -578,7 +579,7 @@ choose_next_subplan_locally(AppendState *node)
}
else if (node->as_valid_subplans == NULL)
node->as_valid_subplans =
- ExecFindMatchingSubPlans(node->as_prune_state, false);
+ ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
whichplan = -1;
}
@@ -643,7 +644,7 @@ choose_next_subplan_for_leader(AppendState *node)
if (node->as_valid_subplans == NULL)
{
node->as_valid_subplans =
- ExecFindMatchingSubPlans(node->as_prune_state, false);
+ ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
/*
* Mark each invalid plan as finished to allow the loop below to
@@ -718,7 +719,7 @@ choose_next_subplan_for_worker(AppendState *node)
else if (node->as_valid_subplans == NULL)
{
node->as_valid_subplans =
- ExecFindMatchingSubPlans(node->as_prune_state, false);
+ ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
mark_invalid_subplans_as_finished(node);
}
@@ -869,7 +870,7 @@ ExecAppendAsyncBegin(AppendState *node)
if (node->as_valid_subplans == NULL)
{
node->as_valid_subplans =
- ExecFindMatchingSubPlans(node->as_prune_state, false);
+ ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
classify_matching_subplans(node);
}
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 399b39c598..c653084515 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -104,7 +104,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
* subplan, we can fill ms_valid_subplans immediately, preventing
* later calls to ExecFindMatchingSubPlans.
*/
- if (!prunestate->do_exec_prune && nplans > 0)
+ if (mergestate->ms_prune_state == NULL ||
+ (!mergestate->ms_prune_state->do_exec_prune && nplans > 0))
mergestate->ms_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
}
else
@@ -219,7 +220,7 @@ ExecMergeAppend(PlanState *pstate)
*/
if (node->ms_valid_subplans == NULL)
node->ms_valid_subplans =
- ExecFindMatchingSubPlans(node->ms_prune_state, false);
+ ExecFindMatchingSubPlans(node->ms_prune_state, false, NULL);
/*
* First time through: pull the first tuple from each valid subplan,
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index b4fa8d90bc..ff363be811 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -372,6 +372,7 @@ set_plan_references(PlannerInfo *root, Plan *plan)
{
PartitionPruneInfo *pruneinfo = lfirst(lc);
ListCell *l;
+ Bitmapset *leafpart_rtis = NULL;
pruneinfo->root_parent_relids =
offset_relid_set(pruneinfo->root_parent_relids, rtoffset);
@@ -383,17 +384,52 @@ set_plan_references(PlannerInfo *root, Plan *plan)
foreach(l2, prune_infos)
{
PartitionedRelPruneInfo *pinfo = lfirst(l2);
+ int i;
/* RT index of the table to which the pinfo belongs. */
pinfo->rtindex += rtoffset;
+
+ /* Also of the leaf partitions that might be scanned. */
+ for (i = 0; i < pinfo->nparts; i++)
+ {
+ if (pinfo->rti_map[i] > 0 && pinfo->subplan_map[i] >= 0)
+ {
+ pinfo->rti_map[i] += rtoffset;
+ leafpart_rtis = bms_add_member(leafpart_rtis,
+ pinfo->rti_map[i]);
+ }
+ }
}
}
+ if (pruneinfo->needs_init_pruning)
+ {
+ glob->containsInitialPruning = true;
+
+ /*
+ * Delete the leaf partition RTIs from the set of relations to be
+ * locked by AcquireExecutorLocks(). The actual set of leaf
+ * partitions to be locked is computed by
+ * ExecLockRelationsIfNeeded().
+ */
+ glob->minLockRelids = bms_del_members(glob->minLockRelids,
+ leafpart_rtis);
+ }
+
glob->partPruneInfos = lappend(glob->partPruneInfos, pruneinfo);
glob->containsInitialPruning |= pruneinfo->needs_init_pruning;
}
+ /*
+ * It seems worth doing a bms_copy() on glob->minLockRelids if we deleted
+ * bits from it above to get rid of any empty tail bits. It seems better
+ * for the loop over this set in AcquireExecutorLocks() to not have to go
+ * through those useless bit words.
+ */
+ if (glob->containsInitialPruning)
+ glob->minLockRelids = bms_copy(glob->minLockRelids);
+
return result;
}
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index f113170140..af5e9b1609 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
*
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
- *
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -803,60 +800,69 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
+ return cplan->is_valid;
+}
- return false;
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor after it has finished taking locks on a plan tree
+ * in a CachedPlan.
+ */
+bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return GenericPlanIsValid(cplan);
}
/*
@@ -1126,9 +1132,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
- *
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
* the refcount has been reported to that ResourceOwner (note that this
@@ -1362,6 +1365,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
/*
* Reject if AcquireExecutorLocks would have anything to do. This is
* probably unnecessary given the previous check, but let's be safe.
+ * XXX - maybe remove?
*/
foreach(lc, plan->stmt_list)
{
@@ -1735,62 +1739,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- Bitmapset *allLockRelids;
- int rti;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- Assert(plannedstmt->minLockRelids == NULL);
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- allLockRelids = plannedstmt->minLockRelids;
- rti = -1;
- while ((rti = bms_next_member(allLockRelids, rti)) > 0)
- {
- RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h
index 21d85a7809..526f5781da 100644
--- a/src/include/executor/execPartition.h
+++ b/src/include/executor/execPartition.h
@@ -133,5 +133,11 @@ extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
bool consider_initial_steps,
bool consider_exec_steps);
extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
- bool initial_prune);
+ bool initial_prune,
+ Bitmapset **scan_leafpart_rtis);
+extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
+ EState *estate,
+ PartitionPruneInfo *pruneinfo,
+ bool consider_initial_steps,
+ bool consider_exec_steps);
#endif /* EXECPARTITION_H */
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index 4b7368a0dc..595297df6c 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -46,6 +46,12 @@ typedef struct QueryDesc
QueryEnvironment *queryEnv; /* query environment passed in */
int instrument_options; /* OR of InstrumentOption flags */
+ /*
+ * Used by ExecParallelGetQueryDesc() to save the result of initial
+ * partition pruning performed by the leader.
+ */
+ List *part_prune_results; /* list of PartitionPruneResult */
+
/* These fields are set by ExecutorStart */
TupleDesc tupDesc; /* descriptor for result tuples */
EState *estate; /* executor's query-wide state */
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 63f3d09804..755e231675 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -59,6 +59,8 @@
#define EXEC_FLAG_MARK 0x0008 /* need mark/restore */
#define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */
#define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */
+#define EXEC_FLAG_GET_LOCKS 0x0400 /* should ExecGetRangeTableRelation()
+ * lock relations? */
/* Hook for plugins to get control in ExecutorStart() */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 20f4c8b35f..b361592e2d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -620,6 +620,7 @@ typedef struct EState
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */
+ List *es_part_prune_results; /* QueryDesc.part_prune_results */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index d00b5dcb03..83e5c665c7 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -134,8 +134,8 @@ typedef struct PlannerGlobal
bool containsInitialPruning;
/*
- * Indexes of all range table entries; for AcquireExecutorLocks()'s
- * perusal.
+ * Indexes of all range table entries except those of leaf partitions
+ * scanned by prunable subplans; for AcquireExecutorLocks() perusal.
*/
Bitmapset *minLockRelids;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7b53f990e0..e76e945c8c 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -82,8 +82,9 @@ typedef struct PlannedStmt
List *permInfos; /* list of RTEPermissionInfo nodes for rtable
* entries needing one */
- Bitmapset *minLockRelids; /* Indexes of all range table entries; for
- * AcquireExecutorLocks()'s perusal */
+ Bitmapset *minLockRelids; /* Indexes of all range table entries except
+ * those of leaf partitions scanned by
+ * prunable subplans */
/* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */
List *resultRelations; /* integer list of RT indexes, or NIL */
@@ -1575,6 +1576,32 @@ typedef struct PartitionPruneStepCombine
List *source_stepids;
} PartitionPruneStepCombine;
+/*----------------
+ * PartitionPruneResult
+ *
+ * The result of performing ExecPartitionDoInitialPruning() on a given
+ * PartitionPruneInfo.
+ *
+ * root_parent_relids is same as PartitionPruneInfo.root_parent_relids. It's
+ * there for cross-checking in ExecInitPartitionPruning() that the
+ * PartitionPruneResult and the PartitionPruneInfo at a given index in
+ * EState.es_part_prune_results and EState.es_part_prune_infos, respectively,
+ * belong to the same parent plan node.
+ *
+ * validsubplans contains the indexes of subplans remaining after performing
+ * initial pruning by calling ExecFindMatchingSubPlans() on the
+ * PartitionPruneInfo.
+ *
+ * This is used to store the result of initial partition pruning that is
+ * peformed in ExecDoInitialPartitionPruning().
+ */
+typedef struct PartitionPruneResult
+{
+ NodeTag type;
+
+ Bitmapset *root_parent_relids;
+ Bitmapset *validsubplans;
+} PartitionPruneResult;
/*
* Plan invalidation info
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index a443181d41..7c664bad35 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,7 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+extern bool CachedPlanStillValid(CachedPlan *cplan);
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
[application/octet-stream] v31-0002-Preparatory-refactoring-before-reworking-CachedP.patch (21.4K, 4-v31-0002-Preparatory-refactoring-before-reworking-CachedP.patch)
download | inline diff:
From 936ef111a9b515c0d0111637a22959ea62e92b5d Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Tue, 13 Dec 2022 11:58:07 +0900
Subject: [PATCH v31 2/3] Preparatory refactoring before reworking CachedPlan
locking
Remember the RT indexes of RTEs that AcquireExecutorLocks() must
look at to consider locking in a bitmapset, so that nstead of looping
over the range table to find those RTEs, it can look them up using
the RT indexes set in the bitmapset.
This also adds some extra information related to execution-time
pruning to the relevant plan nodes.
---
src/backend/executor/execParallel.c | 1 +
src/backend/executor/execPartition.c | 34 +++++++++++++++-------
src/backend/nodes/readfuncs.c | 8 ++++--
src/backend/optimizer/plan/planner.c | 2 ++
src/backend/optimizer/plan/setrefs.c | 12 ++++++++
src/backend/partitioning/partprune.c | 42 ++++++++++++++++++++++++++--
src/backend/utils/cache/plancache.c | 10 +++++--
src/include/executor/execPartition.h | 6 ++++
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 11 ++++++++
src/include/nodes/plannodes.h | 19 +++++++++++++
11 files changed, 128 insertions(+), 18 deletions(-)
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 5f97f5353f..1f5d6d4d64 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -182,6 +182,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt->transientPlan = false;
pstmt->dependsOnRole = false;
pstmt->parallelModeNeeded = false;
+ pstmt->containsInitialPruning = false; /* workers need not know! */
pstmt->planTree = plan;
pstmt->partPruneInfos = estate->es_part_prune_infos;
pstmt->rtable = estate->es_range_table;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 651ad24fc1..4b91bb7403 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -184,8 +184,6 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel,
int maxfieldlen);
static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri);
static List *adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap);
-static PartitionPruneState *CreatePartitionPruneState(PlanState *planstate,
- PartitionPruneInfo *pruneinfo);
static void InitPartitionPruneContext(PartitionPruneContext *context,
List *pruning_steps,
PartitionDesc partdesc,
@@ -1759,6 +1757,11 @@ adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap)
* account for initial pruning possibly having eliminated some of the
* subplans.
*
+ * ExecCreatePartitionPruneState:
+ * A sub-routine of ExecInitPartitionPruning() that creates the
+ * PartitionPruneState from a given PartitionPruneInfo. Exported for the
+ * use by callers that don't need to do ExecInitPartitionPruning().
+ *
* ExecFindMatchingSubPlans:
* Returns indexes of matching subplans after evaluating the expressions
* that are safe to evaluate at a given point. This function is first
@@ -1812,7 +1815,9 @@ ExecInitPartitionPruning(PlanState *planstate,
ExecAssignExprContext(estate, planstate);
/* Create the working data structure for pruning */
- prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ prunestate = ExecCreatePartitionPruneState(planstate, estate, pruneinfo,
+ pruneinfo->needs_init_pruning,
+ pruneinfo->needs_exec_pruning);
/*
* Perform an initial partition prune pass, if required.
@@ -1849,7 +1854,7 @@ ExecInitPartitionPruning(PlanState *planstate,
}
/*
- * CreatePartitionPruneState
+ * ExecCreatePartitionPruneState
* Build the data structure required for calling ExecFindMatchingSubPlans
*
* 'planstate' is the parent plan node's execution state.
@@ -1865,15 +1870,18 @@ ExecInitPartitionPruning(PlanState *planstate,
* re-evaluate which partitions match the pruning steps provided in each
* PartitionedRelPruneInfo.
*/
-static PartitionPruneState *
-CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
+PartitionPruneState *
+ExecCreatePartitionPruneState(PlanState *planstate, EState *estate,
+ PartitionPruneInfo *pruneinfo,
+ bool consider_initial_steps,
+ bool consider_exec_steps)
{
- EState *estate = planstate->state;
PartitionPruneState *prunestate;
int n_part_hierarchies;
ListCell *lc;
int i;
- ExprContext *econtext = planstate->ps_ExprContext;
+ ExprContext *econtext = planstate ? planstate->ps_ExprContext :
+ GetPerTupleExprContext(estate);
/* For data reading, executor always omits detached partitions */
if (estate->es_partition_directory == NULL)
@@ -1955,6 +1963,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
Assert(partdesc->nparts >= pinfo->nparts);
pprune->nparts = partdesc->nparts;
pprune->subplan_map = palloc(sizeof(int) * partdesc->nparts);
+ pprune->rti_map = palloc(sizeof(Index) * partdesc->nparts);
if (partdesc->nparts == pinfo->nparts)
{
/*
@@ -1965,6 +1974,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
pprune->subpart_map = pinfo->subpart_map;
memcpy(pprune->subplan_map, pinfo->subplan_map,
sizeof(int) * pinfo->nparts);
+ memcpy(pprune->rti_map, pinfo->rti_map,
+ sizeof(int) * pinfo->nparts);
/*
* Double-check that the list of unpruned relations has not
@@ -2015,6 +2026,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
pinfo->subplan_map[pd_idx];
pprune->subpart_map[pp_idx] =
pinfo->subpart_map[pd_idx];
+ pprune->rti_map[pp_idx] =
+ pinfo->rti_map[pd_idx];
pd_idx++;
}
else
@@ -2022,6 +2035,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
/* this partdesc entry is not in the plan */
pprune->subplan_map[pp_idx] = -1;
pprune->subpart_map[pp_idx] = -1;
+ pprune->rti_map[pp_idx] = 0;
}
}
@@ -2043,7 +2057,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* Initialize pruning contexts as needed.
*/
pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
- if (pinfo->initial_pruning_steps)
+ if (consider_initial_steps && pinfo->initial_pruning_steps)
{
InitPartitionPruneContext(&pprune->initial_context,
pinfo->initial_pruning_steps,
@@ -2053,7 +2067,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
prunestate->do_initial_prune = true;
}
pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
- if (pinfo->exec_pruning_steps)
+ if (consider_exec_steps && pinfo->exec_pruning_steps)
{
InitPartitionPruneContext(&pprune->exec_context,
pinfo->exec_pruning_steps,
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index f3629cdfd1..caf2a60493 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -158,6 +158,11 @@
token = pg_strtok(&length); /* skip :fldname */ \
local_node->fldname = readIntCols(len)
+/* Read an Index array */
+#define READ_INDEX_ARRAY(fldname, len) \
+ token = pg_strtok(&length); /* skip :fldname */ \
+ local_node->fldname = readIndexCols(len)
+
/* Read a bool array */
#define READ_BOOL_ARRAY(fldname, len) \
token = pg_strtok(&length); /* skip :fldname */ \
@@ -800,7 +805,6 @@ fnname(int numCols) \
*/
READ_SCALAR_ARRAY(readAttrNumberCols, int16, atoi)
READ_SCALAR_ARRAY(readOidCols, Oid, atooid)
-/* outfuncs.c has writeIndexCols, but we don't yet need that here */
-/* READ_SCALAR_ARRAY(readIndexCols, Index, atoui) */
+READ_SCALAR_ARRAY(readIndexCols, Index, atoui)
READ_SCALAR_ARRAY(readIntCols, int, atoi)
READ_SCALAR_ARRAY(readBoolCols, bool, strtobool)
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 05f44faf6e..2b7238cf24 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -525,8 +525,10 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->parallelModeNeeded = glob->parallelModeNeeded;
result->planTree = top_plan;
result->partPruneInfos = glob->partPruneInfos;
+ result->containsInitialPruning = glob->containsInitialPruning;
result->rtable = glob->finalrtable;
result->permInfos = glob->finalrteperminfos;
+ result->minLockRelids = glob->minLockRelids;
result->resultRelations = glob->resultRelations;
result->appendRelations = glob->appendRelations;
result->subplans = glob->subplans;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 85ba9d1ca1..b4fa8d90bc 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -279,6 +279,16 @@ set_plan_references(PlannerInfo *root, Plan *plan)
*/
add_rtes_to_flat_rtable(root, false);
+ /*
+ * Add the query's adjusted range of RT indexes to glob->minLockRelids.
+ * The adjusted RT indexes of prunable relations will be deleted from the
+ * set below where PartitionPruneInfos are processed.
+ */
+ glob->minLockRelids =
+ bms_add_range(glob->minLockRelids,
+ rtoffset + 1,
+ rtoffset + list_length(root->parse->rtable));
+
/*
* Adjust RT indexes of PlanRowMarks and add to final rowmarks list
*/
@@ -377,9 +387,11 @@ set_plan_references(PlannerInfo *root, Plan *plan)
/* RT index of the table to which the pinfo belongs. */
pinfo->rtindex += rtoffset;
}
+
}
glob->partPruneInfos = lappend(glob->partPruneInfos, pruneinfo);
+ glob->containsInitialPruning |= pruneinfo->needs_init_pruning;
}
return result;
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 510145e3c0..9ae41053da 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -144,7 +144,9 @@ static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
List *prunequal,
Bitmapset *partrelids,
int *relid_subplan_map,
- Bitmapset **matchedsubplans);
+ Bitmapset **matchedsubplans,
+ bool *needs_init_pruning,
+ bool *needs_exec_pruning);
static void gen_partprune_steps(RelOptInfo *rel, List *clauses,
PartClauseTarget target,
GeneratePruningStepsContext *context);
@@ -234,6 +236,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
int *relid_subplan_map;
ListCell *lc;
int i;
+ bool needs_init_pruning = false;
+ bool needs_exec_pruning = false;
/*
* Scan the subpaths to see which ones are scans of partition child
@@ -313,12 +317,16 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Bitmapset *partrelids = (Bitmapset *) lfirst(lc);
List *pinfolist;
Bitmapset *matchedsubplans = NULL;
+ bool partrel_needs_init_pruning;
+ bool partrel_needs_exec_pruning;
pinfolist = make_partitionedrel_pruneinfo(root, parentrel,
prunequal,
partrelids,
relid_subplan_map,
- &matchedsubplans);
+ &matchedsubplans,
+ &partrel_needs_init_pruning,
+ &partrel_needs_exec_pruning);
/* When pruning is possible, record the matched subplans */
if (pinfolist != NIL)
@@ -327,6 +335,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
allmatchedsubplans = bms_join(matchedsubplans,
allmatchedsubplans);
}
+
+ needs_init_pruning |= partrel_needs_init_pruning;
+ needs_exec_pruning |= partrel_needs_exec_pruning;
}
pfree(relid_subplan_map);
@@ -342,6 +353,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
pruneinfo = makeNode(PartitionPruneInfo);
pruneinfo->root_parent_relids = parentrel->relids;
pruneinfo->prune_infos = prunerelinfos;
+ pruneinfo->needs_init_pruning = needs_init_pruning;
+ pruneinfo->needs_exec_pruning = needs_exec_pruning;
/*
* Some subplans may not belong to any of the identified partitioned rels.
@@ -442,13 +455,19 @@ add_part_relids(List *allpartrelids, Bitmapset *partrelids)
* If we cannot find any useful run-time pruning steps, return NIL.
* However, on success, each rel identified in partrelids will have
* an element in the result list, even if some of them are useless.
+ * *needs_init_pruning and *needs_exec_pruning are set to indicate whether
+ * the pruning steps contained in the returned PartitionedRelPruneInfos
+ * can be performed during executor startup and during execution,
+ * respectively.
*/
static List *
make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *prunequal,
Bitmapset *partrelids,
int *relid_subplan_map,
- Bitmapset **matchedsubplans)
+ Bitmapset **matchedsubplans,
+ bool *needs_init_pruning,
+ bool *needs_exec_pruning)
{
RelOptInfo *targetpart = NULL;
List *pinfolist = NIL;
@@ -459,6 +478,10 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
int rti;
int i;
+ /* Will find out below. */
+ *needs_init_pruning = false;
+ *needs_exec_pruning = false;
+
/*
* Examine each partitioned rel, constructing a temporary array to map
* from planner relids to index of the partitioned rel, and building a
@@ -546,6 +569,9 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* executor per-scan pruning steps. This first pass creates startup
* pruning steps and detects whether there's any possibly-useful quals
* that would require per-scan pruning.
+ *
+ * In the first pass, we note whether the 2nd pass is necessary by
+ * noting the presence of EXEC parameters.
*/
gen_partprune_steps(subpart, partprunequal, PARTTARGET_INITIAL,
&context);
@@ -620,6 +646,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
pinfo->execparamids = execparamids;
/* Remaining fields will be filled in the next loop */
+ /* record which types of pruning steps we've seen so far */
+ if (initial_pruning_steps != NIL)
+ *needs_init_pruning = true;
+ if (exec_pruning_steps != NIL)
+ *needs_exec_pruning = true;
+
pinfolist = lappend(pinfolist, pinfo);
}
@@ -647,6 +679,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
int *subplan_map;
int *subpart_map;
Oid *relid_map;
+ Index *rti_map;
/*
* Construct the subplan and subpart maps for this partitioning level.
@@ -659,6 +692,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
subpart_map = (int *) palloc(nparts * sizeof(int));
memset(subpart_map, -1, nparts * sizeof(int));
relid_map = (Oid *) palloc0(nparts * sizeof(Oid));
+ rti_map = (Index *) palloc0(nparts * sizeof(Index));
present_parts = NULL;
i = -1;
@@ -673,6 +707,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
subplan_map[i] = subplanidx = relid_subplan_map[partrel->relid] - 1;
subpart_map[i] = subpartidx = relid_subpart_map[partrel->relid] - 1;
relid_map[i] = planner_rt_fetch(partrel->relid, root)->relid;
+ rti_map[i] = partrel->relid;
if (subplanidx >= 0)
{
present_parts = bms_add_member(present_parts, i);
@@ -697,6 +732,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
pinfo->subplan_map = subplan_map;
pinfo->subpart_map = subpart_map;
pinfo->relid_map = relid_map;
+ pinfo->rti_map = rti_map;
}
pfree(relid_subpart_map);
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 77c2ba3f8f..f113170140 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -1747,7 +1747,8 @@ AcquireExecutorLocks(List *stmt_list, bool acquire)
foreach(lc1, stmt_list)
{
PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
+ Bitmapset *allLockRelids;
+ int rti;
if (plannedstmt->commandType == CMD_UTILITY)
{
@@ -1760,14 +1761,17 @@ AcquireExecutorLocks(List *stmt_list, bool acquire)
*/
Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
+ Assert(plannedstmt->minLockRelids == NULL);
if (query)
ScanQueryForLocks(query, acquire);
continue;
}
- foreach(lc2, plannedstmt->rtable)
+ allLockRelids = plannedstmt->minLockRelids;
+ rti = -1;
+ while ((rti = bms_next_member(allLockRelids, rti)) > 0)
{
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
+ RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable);
if (!(rte->rtekind == RTE_RELATION ||
(rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h
index ee487e42dd..21d85a7809 100644
--- a/src/include/executor/execPartition.h
+++ b/src/include/executor/execPartition.h
@@ -45,6 +45,7 @@ extern void ExecCleanupTupleRouting(ModifyTableState *mtstate,
* nparts Length of subplan_map[] and subpart_map[].
* subplan_map Subplan index by partition index, or -1.
* subpart_map Subpart index by partition index, or -1.
+ * rti_map Range table index by partition index, or 0.
* present_parts A Bitmapset of the partition indexes that we
* have subplans or subparts for.
* initial_pruning_steps List of PartitionPruneSteps used to
@@ -61,6 +62,7 @@ typedef struct PartitionedRelPruningData
int nparts;
int *subplan_map;
int *subpart_map;
+ Index *rti_map;
Bitmapset *present_parts;
List *initial_pruning_steps;
List *exec_pruning_steps;
@@ -126,6 +128,10 @@ extern PartitionPruneState *ExecInitPartitionPruning(PlanState *planstate,
int part_prune_index,
Bitmapset *root_parent_relids,
Bitmapset **initially_valid_subplans);
+extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate, EState *estate,
+ PartitionPruneInfo *pruneinfo,
+ bool consider_initial_steps,
+ bool consider_exec_steps);
extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
bool initial_prune);
#endif /* EXECPARTITION_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10752e8011..1de8f3fadc 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -218,6 +218,7 @@ extern struct Bitmapset *readBitmapset(void);
extern uintptr_t readDatum(bool typbyval);
extern bool *readBoolCols(int numCols);
extern int *readIntCols(int numCols);
+extern Index *readIndexCols(int numCols);
extern Oid *readOidCols(int numCols);
extern int16 *readAttrNumberCols(int numCols);
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2d1d8f4bcd..d00b5dcb03 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -128,6 +128,17 @@ typedef struct PlannerGlobal
/* List of PartitionPruneInfo contained in the plan */
List *partPruneInfos;
+ /*
+ * Do any of those PartitionPruneInfos have initial pruning steps in them?
+ */
+ bool containsInitialPruning;
+
+ /*
+ * Indexes of all range table entries; for AcquireExecutorLocks()'s
+ * perusal.
+ */
+ Bitmapset *minLockRelids;
+
/* OIDs of relations the plan depends on */
List *relationOids;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index c1234fcf36..7b53f990e0 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -73,11 +73,18 @@ typedef struct PlannedStmt
List *partPruneInfos; /* List of PartitionPruneInfo contained in the
* plan */
+ bool containsInitialPruning; /* Do any of those PartitionPruneInfos
+ * have initial pruning steps in them?
+ */
+
List *rtable; /* list of RangeTblEntry nodes */
List *permInfos; /* list of RTEPermissionInfo nodes for rtable
* entries needing one */
+ Bitmapset *minLockRelids; /* Indexes of all range table entries; for
+ * AcquireExecutorLocks()'s perusal */
+
/* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */
List *resultRelations; /* integer list of RT indexes, or NIL */
@@ -1417,6 +1424,13 @@ typedef struct PlanRowMark
* prune_infos List of Lists containing PartitionedRelPruneInfo nodes,
* one sublist per run-time-prunable partition hierarchy
* appearing in the parent plan node's subplans.
+ *
+ * needs_init_pruning Does any of the PartitionedRelPruneInfos in
+ * prune_infos have its initial_pruning_steps set?
+ *
+ * needs_exec_pruning Does any of the PartitionedRelPruneInfos in
+ * prune_infos have its exec_pruning_steps set?
+ *
* other_subplans Indexes of any subplans that are not accounted for
* by any of the PartitionedRelPruneInfo nodes in
* "prune_infos". These subplans must not be pruned.
@@ -1428,6 +1442,8 @@ typedef struct PartitionPruneInfo
NodeTag type;
Bitmapset *root_parent_relids;
List *prune_infos;
+ bool needs_init_pruning;
+ bool needs_exec_pruning;
Bitmapset *other_subplans;
} PartitionPruneInfo;
@@ -1472,6 +1488,9 @@ typedef struct PartitionedRelPruneInfo
/* relation OID by partition index, or 0 */
Oid *relid_map pg_node_attr(array_size(nparts));
+ /* Range table index by partition index, or 0. */
+ Index *rti_map pg_node_attr(array_size(nparts));
+
/*
* initial_pruning_steps shows how to prune during executor startup (i.e.,
* without use of any PARAM_EXEC Params); it is NIL if no startup pruning
--
2.35.3
view thread (22+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: generic plans and "initial" pruning
In-Reply-To: <CA+HiwqEDr9m3NGrmiOgatCnRPwD95=MHgWQdwvnoMyQd3k9-Yw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox