public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Langote <[email protected]>
To: Robert Haas <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Cc: David Rowley *EXTERN* <[email protected]>
Subject: Re: generic plans and "initial" pruning
Date: Fri, 11 Mar 2022 23:35:37 +0900
Message-ID: <CA+HiwqEYCpEqh2LMDOp9mT+4-QoVe8HgFMKBjntEMCTZLpcCCA@mail.gmail.com> (raw)
In-Reply-To: <CA+HiwqGbnNGttKr8fK2pAYcC1JAapo=1nXO7OQs2M7+HmoVVpw@mail.gmail.com>
References: <CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com>
<CA+TgmoZ=92LxPeVJ20vf4vJ4o8dr7Q2seDQR=xLY04puLjDs_A@mail.gmail.com>
<CA+HiwqEAdv+1t+v6pkcH9pgsSXnBeHLchDuGRwXP-oEB4BCPmg@mail.gmail.com>
<CA+TgmobN=+AoDE4JZioVpaRXJVCQ2Fa9Uw3TjC-OqnXWQ0uq1w@mail.gmail.com>
<CA+HiwqFOsJ21nmvoKtPkzxKfYzACajj-mAOoJ+-y_O6g+-v-aA@mail.gmail.com>
<CA+TgmoanT5K44bHZqMXxjAKwyF7nhajKBrw-p-+HSyEzOVqX1w@mail.gmail.com>
<CA+HiwqGbnNGttKr8fK2pAYcC1JAapo=1nXO7OQs2M7+HmoVVpw@mail.gmail.com>
On Mon, Mar 7, 2022 at 11:18 PM Amit Langote <[email protected]> wrote:
> On Fri, Feb 11, 2022 at 7:02 AM Robert Haas <[email protected]> wrote:
> > I don't love PlanPrepOutput the way you have it. I think one of the
> > basic design issues for this patch is: should we think of the prep
> > phase as specifically pruning, or is it general prep and pruning is
> > the first thing for which we're going to use it? If it's really a
> > pre-pruning phase, we could name it that way instead of calling it
> > "prep". If it's really a general prep phase, then why does
> > PlanPrepOutput contain initially_valid_subnodes as a field? One could
> > imagine letting each prep function decide what kind of prep node it
> > would like to return, with partition pruning being just one of the
> > options. But is that a useful generalization of the basic concept, or
> > just pretending that a special-purpose mechanism is more general than
> > it really is?
>
> While it can feel like the latter TBH, I'm inclined to keep
> ExecutorPrep generalized. What bothers me about about the
> alternative of calling the new phase something less generalized like
> ExecutorDoInitPruning() is that that makes the somewhat elaborate API
> changes needed for the phase's output to put into QueryDesc, through
> which it ultimately reaches the main executor, seem less worthwhile.
>
> I agree that PlanPrepOutput design needs to be likewise generalized,
> maybe like you suggest -- using PlanInitPruningOutput, a child class
> of PlanPrepOutput, to return the prep output for plan nodes that
> support pruning.
>
> Thoughts?
So I decided to agree with you after all about limiting the scope of
this new executor interface, or IOW call it what it is.
I have named it ExecutorGetLockRels() to go with the only use case we
know for it -- get the set of relations for AcquireExecutorLocks() to
lock to validate a plan tree. Its result returned in a node named
ExecLockRelsInfo, which contains the set of relations scanned in the
plan tree (lockrels) and a list of PlanInitPruningOutput nodes for all
nodes that undergo pruning.
> > + return CreateQueryDesc(pstmt, NULL, /* XXX pass ExecPrepOutput too? */
> >
> > It seems to me that we should do what the XXX suggests. It doesn't
> > seem nice if the parallel workers could theoretically decide to prune
> > a different set of nodes than the leader.
>
> OK, will fix.
Done. This required adding nodeToString() and stringToNode() support
for the nodes produced by the new executor function that wasn't there
before.
> > Somewhere, we're going to need to document the idea that this may
> > permit us to execute a plan that isn't actually fully valid, but that
> > we expect to survive because we'll never do anything with the parts of
> > it that aren't. Maybe that should be added to the executor README, or
> > maybe there's some better place, but I don't think that should remain
> > something that's just implicit.
>
> Agreed. I'd added a description of the new prep phase to executor
> README, though the text didn't mention this particular bit. Will fix
> to mention it.
Rewrote the comments above ExecutorGetLockRels() (previously
ExecutorPrep()) and the executor README text to be explicit about the
fact that not locking some relations effectively invalidates pruned
parts of the plan tree.
> > This is not a full review, just some initial thoughts looking through this.
>
> Thanks again. Will post a new version soon after a bit more polishing.
Attached is v5, now broken into 3 patches:
0001: Some refactoring of runtime pruning code
0002: Add a plan_tree_walker
0003: Teach AcquireExecutorLocks to skip locking pruned relations
--
Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v5-0002-Add-a-plan_tree_walker.patch (3.9K, 2-v5-0002-Add-a-plan_tree_walker.patch)
download | inline diff:
From 22ff31c7b052eabb32f4a529c48fe48180332156 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Thu, 3 Mar 2022 16:04:13 +0900
Subject: [PATCH v5 2/3] Add a plan_tree_walker()
Like planstate_tree_walker() but for uninitialized plan trees.
---
src/backend/nodes/nodeFuncs.c | 116 ++++++++++++++++++++++++++++++++++
src/include/nodes/nodeFuncs.h | 3 +
2 files changed, 119 insertions(+)
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 47d0564fa2..cdf937f127 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -31,6 +31,10 @@ static bool planstate_walk_subplans(List *plans, bool (*walker) (),
void *context);
static bool planstate_walk_members(PlanState **planstates, int nplans,
bool (*walker) (), void *context);
+static bool plan_walk_subplans(List *plans,
+ bool (*walker) (),
+ void *context);
+static bool plan_walk_members(List *plans, bool (*walker) (), void *context);
/*
@@ -4148,3 +4152,115 @@ planstate_walk_members(PlanState **planstates, int nplans,
return false;
}
+
+/*
+ * plan_tree_walker --- walk plantrees
+ *
+ * The walker has already visited the current node, and so we need only
+ * recurse into any sub-nodes it has.
+ */
+bool
+plan_tree_walker(Plan *plan,
+ bool (*walker) (),
+ void *context)
+{
+ /* Guard against stack overflow due to overly complex plan trees */
+ check_stack_depth();
+
+ /* initPlan-s */
+ if (plan_walk_subplans(plan->initPlan, walker, context))
+ return true;
+
+ /* lefttree */
+ if (outerPlan(plan))
+ {
+ if (walker(outerPlan(plan), context))
+ return true;
+ }
+
+ /* righttree */
+ if (innerPlan(plan))
+ {
+ if (walker(innerPlan(plan), context))
+ return true;
+ }
+
+ /* special child plans */
+ switch (nodeTag(plan))
+ {
+ case T_Append:
+ if (plan_walk_members(((Append *) plan)->appendplans,
+ walker, context))
+ return true;
+ break;
+ case T_MergeAppend:
+ if (plan_walk_members(((MergeAppend *) plan)->mergeplans,
+ walker, context))
+ return true;
+ break;
+ case T_BitmapAnd:
+ if (plan_walk_members(((BitmapAnd *) plan)->bitmapplans,
+ walker, context))
+ return true;
+ break;
+ case T_BitmapOr:
+ if (plan_walk_members(((BitmapOr *) plan)->bitmapplans,
+ walker, context))
+ return true;
+ break;
+ case T_CustomScan:
+ if (plan_walk_members(((CustomScan *) plan)->custom_plans,
+ walker, context))
+ return true;
+ break;
+ case T_SubqueryScan:
+ if (walker(((SubqueryScan *) plan)->subplan, context))
+ return true;
+ break;
+ default:
+ break;
+ }
+
+ return false;
+}
+
+/*
+ * Walk a list of SubPlans (or initPlans, which also use SubPlan nodes).
+ */
+static bool
+plan_walk_subplans(List *plans,
+ bool (*walker) (),
+ void *context)
+{
+ ListCell *lc;
+ PlannedStmt *plannedstmt = (PlannedStmt *) context;
+
+ foreach(lc, plans)
+ {
+ SubPlan *sp = lfirst_node(SubPlan, lc);
+ Plan *p = list_nth(plannedstmt->subplans, sp->plan_id - 1);
+
+ if (walker(p, context))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Walk the constituent plans of a ModifyTable, Append, MergeAppend,
+ * BitmapAnd, or BitmapOr node.
+ */
+static bool
+plan_walk_members(List *plans, bool (*walker) (), void *context)
+{
+ ListCell *lc;
+
+ foreach(lc, plans)
+ {
+ if (walker(lfirst(lc), context))
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/include/nodes/nodeFuncs.h b/src/include/nodes/nodeFuncs.h
index 93c60bde66..fca107ad65 100644
--- a/src/include/nodes/nodeFuncs.h
+++ b/src/include/nodes/nodeFuncs.h
@@ -158,5 +158,8 @@ extern bool raw_expression_tree_walker(Node *node, bool (*walker) (),
struct PlanState;
extern bool planstate_tree_walker(struct PlanState *planstate, bool (*walker) (),
void *context);
+struct Plan;
+extern bool plan_tree_walker(struct Plan *plan, bool (*walker) (),
+ void *context);
#endif /* NODEFUNCS_H */
--
2.24.1
[application/octet-stream] v5-0003-Teach-AcquireExecutorLocks-to-skip-locking-pruned.patch (93.2K, 3-v5-0003-Teach-AcquireExecutorLocks-to-skip-locking-pruned.patch)
download | inline diff:
From 62fd8ca887f62dcd89010bf4475529eb16f07d52 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Wed, 22 Dec 2021 16:55:17 +0900
Subject: [PATCH v5 3/3] Teach AcquireExecutorLocks() to skip locking pruned
partitions
Instead of locking all relations listed in the range table, this
asks the new executor function ExecutorGetLockRels() to return a set
of relations (their RT indexes) to lock or simply use the set
given by PlannedStmt.lockrels. To wit, ExecutorGetLockRels() must be
called if some nodes in the plan tree contain initial pruning steps
(pruning steps containing expressions that can be computed before
before the executor proper has started), which results in the lockrels
set to be computed such that any subplans that are pruned as result of
doing initial pruning do not contribute any relations to the set.
That can result in a much smaller lockrels set when the plan contains
thousands of child subplans, of which only a small number remain
after pruning.
The result of doing the initial pruning during ExecutorGetLockRels()
is preserved for use later during actual execution by creating a
a new node called PlanInitPruningOutput for each plan node that
undergoes pruning and a set of those for the whole plan tree are
put into another new node ExecLockRelsInfo that represents the output
of a given ExecutorGetLockRels() invocation. ExecLockRelsInfos are
passed down the executor alongside the PlannedStmts. This
arrangement ensures that the set of plan tree nodes that
AcquireExecutorLocks() has acquired locks to protect and the one
that the executor will initialize and execute are one and the same.
---
src/backend/commands/copyto.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 7 +-
src/backend/commands/extension.c | 13 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/portalcmds.c | 1 +
src/backend/commands/prepare.c | 17 +-
src/backend/executor/README | 22 ++-
src/backend/executor/execMain.c | 181 +++++++++++++++++++
src/backend/executor/execParallel.c | 27 ++-
src/backend/executor/execPartition.c | 233 +++++++++++++++++++++----
src/backend/executor/execUtils.c | 8 +
src/backend/executor/functions.c | 2 +-
src/backend/executor/nodeAppend.c | 42 ++++-
src/backend/executor/nodeMergeAppend.c | 42 ++++-
src/backend/executor/nodeModifyTable.c | 24 +++
src/backend/executor/spi.c | 14 +-
src/backend/nodes/copyfuncs.c | 50 +++++-
src/backend/nodes/outfuncs.c | 41 +++++
src/backend/nodes/readfuncs.c | 38 ++++
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 10 ++
src/backend/partitioning/partprune.c | 37 +++-
src/backend/tcop/postgres.c | 15 +-
src/backend/tcop/pquery.c | 21 ++-
src/backend/utils/cache/plancache.c | 220 +++++++++++++++++++----
src/backend/utils/mmgr/portalmem.c | 2 +
src/include/commands/explain.h | 3 +-
src/include/executor/execPartition.h | 2 +
src/include/executor/execdesc.h | 2 +
src/include/executor/executor.h | 2 +
src/include/executor/nodeAppend.h | 1 +
src/include/executor/nodeMergeAppend.h | 1 +
src/include/executor/nodeModifyTable.h | 1 +
src/include/nodes/execnodes.h | 87 +++++++++
src/include/nodes/nodes.h | 5 +
src/include/nodes/pathnodes.h | 7 +
src/include/nodes/plannodes.h | 18 ++
src/include/tcop/tcopprot.h | 2 +-
src/include/utils/plancache.h | 5 +
src/include/utils/portal.h | 5 +
41 files changed, 1108 insertions(+), 109 deletions(-)
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 55c38b04c4..d403eb2309 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -542,7 +542,7 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b555..f6607f2454 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index de81379da3..a9dc6d1755 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -407,7 +407,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
}
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(plan, NULL, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
@@ -515,7 +515,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(PlannedStmt *plannedstmt, ExecLockRelsInfo *execlockrelsinfo,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
@@ -563,7 +564,7 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
dest = None_Receiver;
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
+ queryDesc = CreateQueryDesc(plannedstmt, execlockrelsinfo, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 1013790dbb..008b8ce0e9 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -741,8 +741,10 @@ execute_sql_string(const char *sql)
RawStmt *parsetree = lfirst_node(RawStmt, lc1);
MemoryContext per_parsetree_context,
oldcontext;
- List *stmt_list;
- ListCell *lc2;
+ List *stmt_list,
+ *execlockrelsinfo_list;
+ ListCell *lc2,
+ *lc3;
/*
* We do the work for each parsetree in a short-lived context, to
@@ -762,11 +764,13 @@ execute_sql_string(const char *sql)
NULL,
0,
NULL);
- stmt_list = pg_plan_queries(stmt_list, sql, CURSOR_OPT_PARALLEL_OK, NULL);
+ stmt_list = pg_plan_queries(stmt_list, sql, CURSOR_OPT_PARALLEL_OK, NULL,
+ &execlockrelsinfo_list);
- foreach(lc2, stmt_list)
+ forboth(lc2, stmt_list, lc3, execlockrelsinfo_list)
{
PlannedStmt *stmt = lfirst_node(PlannedStmt, lc2);
+ ExecLockRelsInfo *execlockrelsinfo = lfirst_node(ExecLockRelsInfo, lc3);
CommandCounterIncrement();
@@ -777,6 +781,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ execlockrelsinfo,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 05e7b60059..4ef44aaf23 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -416,7 +416,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 9902c5c566..85e73ddded 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -107,6 +107,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
queryString,
CMDTAG_SELECT, /* cursor's query is always a SELECT */
list_make1(plan),
+ list_make1(NULL), /* no ExecLockRelsInfo to pass */
NULL);
/*----------
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 80738547ed..bbbf8bbcbd 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -155,6 +155,7 @@ ExecuteQuery(ParseState *pstate,
PreparedStatement *entry;
CachedPlan *cplan;
List *plan_list;
+ List *plan_execlockrelsinfo_list;
ParamListInfo paramLI = NULL;
EState *estate = NULL;
Portal portal;
@@ -195,6 +196,7 @@ ExecuteQuery(ParseState *pstate,
/* Replan if needed, and increment plan refcount for portal */
cplan = GetCachedPlan(entry->plansource, paramLI, NULL, NULL);
plan_list = cplan->stmt_list;
+ plan_execlockrelsinfo_list = cplan->execlockrelsinfo_list;
/*
* DO NOT add any logic that could possibly throw an error between
@@ -204,7 +206,7 @@ ExecuteQuery(ParseState *pstate,
NULL,
query_string,
entry->plansource->commandTag,
- plan_list,
+ plan_list, plan_execlockrelsinfo_list,
cplan);
/*
@@ -576,7 +578,9 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
const char *query_string;
CachedPlan *cplan;
List *plan_list;
- ListCell *p;
+ List *plan_execlockrelsinfo_list;
+ ListCell *p,
+ *pe;
ParamListInfo paramLI = NULL;
EState *estate = NULL;
instr_time planstart;
@@ -632,15 +636,18 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
plan_list = cplan->stmt_list;
+ plan_execlockrelsinfo_list = cplan->execlockrelsinfo_list;
/* Explain each query */
- foreach(p, plan_list)
+ forboth(p, plan_list, pe, plan_execlockrelsinfo_list)
{
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
+ ExecLockRelsInfo *execlockrelsinfo = lfirst_node(ExecLockRelsInfo, pe);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ ExplainOnePlan(pstmt, execlockrelsinfo, into, es, query_string,
+ paramLI, queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/README b/src/backend/executor/README
index bf5e70860d..27341a2818 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -59,11 +59,20 @@ state tree. Read-only plan trees make life much simpler for plan caching and
reuse.
A corresponding executor state node may not be created during executor startup
-if the executor determines that an entire subplan is not required due to
-execution time partition pruning determining that no matching records will be
-found there. This currently only occurs for Append and MergeAppend nodes. In
-this case the non-required subplans are ignored and the executor state's
-subnode array will become out of sequence to the plan's subplan list.
+if the ExecutorGetLockRels() determines that an entire subplan is not required
+due to initial partition pruning determining that no matching records will be
+found there, while also skipping the locking of relation(s) that would be
+scanned by the subplan were it not pruned. This currently only occurs for
+Append and MergeAppend nodes (see ExecGet[Merge]AppendLockRels()). In this
+case, the non-required subplans are ignored and the executor state's subnode
+array will become out of sequence to the plan's subplan list.
+ExecutorGetLockRels() typically runs before the execution starts, for example,
+as part of checking if a cached generic plan is still valid, though the
+result it produces (ExecLockRelsInfo) is made available to ExecutorStart() via
+the QueryDesc. ExecInitNode() on the plan nodes whose child subplans may have
+been pruned as part of ExecutorGetLockRels() must look up the surviving set of
+subplans to initialize in the ExecLockRelsInfo, instead of reiterating the
+initial pruning computation.
Each Plan node may have expression trees associated with it, to represent
its target list, qualification conditions, etc. These trees are also
@@ -247,6 +256,9 @@ Query Processing Control Flow
This is a sketch of control flow for full query processing:
+ [ ExecutorGetLockRels ] --- an optional step to walk over the plan tree
+ to produce an ExecLockRelsInfo to be passed to CreateQueryDesc
+
CreateQueryDesc
ExecutorStart
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 549d9eb696..3b1f588321 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -48,11 +48,15 @@
#include "commands/matview.h"
#include "commands/trigger.h"
#include "executor/execdebug.h"
+#include "executor/nodeAppend.h"
+#include "executor/nodeMergeAppend.h"
+#include "executor/nodeModifyTable.h"
#include "executor/nodeSubplan.h"
#include "foreign/fdwapi.h"
#include "jit/jit.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "nodes/nodeFuncs.h"
#include "parser/parsetree.h"
#include "storage/bufmgr.h"
#include "storage/lmgr.h"
@@ -100,9 +104,184 @@ static char *ExecBuildSlotValueDescription(Oid reloid,
Bitmapset *modifiedCols,
int maxfieldlen);
static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
+static bool ExecGetScanLockRels(Scan *scan, ExecGetLockRelsContext *context);
/* end of local decls */
+/* ----------------------------------------------------------------
+ * ExecutorGetLockRels
+ *
+ * Figure out the set of relations to lock to be able to execute a given
+ * plan, after taking into account the result of performing any initial
+ * pruning steps present in the plan. Performing those pruning steps
+ * would effectively invalidate the pruned subplans (that is, will not
+ * be looked at during the actual execution of the parent plan), so the
+ * relations that those subplans scan need not be locked.
+ *
+ * Along with the set of RT indexes of relations that must be locked, the
+ * returned struct also contains the information look up PlanInitPruningOutput
+ * nodes, containing the result of performing initial pruning (identities of
+ * surviving partition subnodes), for each plan node that undergoes pruning.
+ *
+ * The caller must arrange to pass on the returned struct down to the
+ * executor, so that the latter can reuse the result of initial pruning to
+ * initialize the same set of surviving subplans, instead of doing the pruning
+ * again by itself.
+ *
+ * This locks relations whose information is perused to do the pruning. For
+ * example, a partitioned table before perusing its PartitionedRelPruneInfo
+ * contained in an Append node to do pruning in ExecGetAppendLockRels().
+ */
+ExecLockRelsInfo *
+ExecutorGetLockRels(PlannedStmt *plannedstmt, ParamListInfo params)
+{
+ int numPlanNodes = plannedstmt->numPlanNodes;
+ ExecGetLockRelsContext context;
+ ExecLockRelsInfo *result;
+ ListCell *lc;
+
+ /* Only get here if there is any pruning to do. */
+ Assert(plannedstmt->containsInitialPruning);
+
+ context.stmt = plannedstmt;
+ context.params = params;
+
+ /* Go do init pruning and fill lockrels. */
+ context.lockrels = NULL;
+ context.initPruningOutputs = NIL;
+ context.ipoIndexes = palloc0(sizeof(int) * numPlanNodes);
+ foreach(lc, plannedstmt->subplans)
+ {
+ Plan *subplan = lfirst(lc);
+
+ (void) ExecGetLockRels(subplan, &context);
+ }
+
+ (void) ExecGetLockRels(plannedstmt->planTree, &context);
+
+ result = makeNode(ExecLockRelsInfo);
+ result->lockrels = context.lockrels;
+ result->numPlanNodes = numPlanNodes;
+ result->initPruningOutputs = context.initPruningOutputs;
+ result->ipoIndexes = context.ipoIndexes;
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * ExecGetLockRels
+ * Recursively find relations to lock in the plan tree rooted at 'node',
+ * performing initial pruning if the node contains the information to
+ * do so
+ *
+ * 'node' is the current node of the plan produced by the query planner
+ * 'context' contains the PlannedStmt and the information about EXTERN
+ * parameters to use for partition pruning and also where to add the
+ * result -- lockrels and PlanInitPruningOutput nodes
+ *
+ * NOTE: ExecGetLockRels subroutine for a given node must add the RT indexes of
+ * any relations that it manipulates to result->lockrels. If the node needs
+ * initial pruning, it must add the resulting PlanInitPruningOutput node to
+ * context using the ExecStorePlanInitPruningOutput() macro.
+ * ------------------------------------------------------------------------
+ */
+bool
+ExecGetLockRels(Plan *node, ExecGetLockRelsContext *context)
+{
+ /* Do nothing when we get to the end of a leaf on tree. */
+ if (node == NULL)
+ return true;
+
+ /* Make sure there's enough stack available. */
+ check_stack_depth();
+
+ switch (nodeTag(node))
+ {
+ case T_Append:
+ if (ExecGetAppendLockRels((Append *) node, context))
+ return true;
+ break;
+ case T_MergeAppend:
+ if (ExecGetMergeAppendLockRels((MergeAppend *) node, context))
+ return true;
+ break;
+
+ case T_SeqScan:
+ case T_SampleScan:
+ case T_IndexScan:
+ case T_IndexOnlyScan:
+ case T_BitmapIndexScan:
+ case T_BitmapHeapScan:
+ case T_TidScan:
+ case T_TidRangeScan:
+ case T_ForeignScan:
+ case T_SubqueryScan:
+ case T_CustomScan:
+ if (ExecGetScanLockRels((Scan *) node, context))
+ return true;
+ break;
+
+ case T_ModifyTable:
+ if (ExecGetModifyTableLockRels((ModifyTable *) node, context))
+ return true;
+ /* plan_tree_walker() will visit the subplan (outerNode) */
+ break;
+
+ default:
+ break;
+ }
+
+ return plan_tree_walker(node, ExecGetLockRels, (void *) context);
+}
+
+/*
+ * ExecGetScanLockRels
+ * Do ExecGetLockRels()'s work for a Scan plan
+ */
+static bool
+ExecGetScanLockRels(Scan *scan, ExecGetLockRelsContext *context)
+{
+ switch (nodeTag(scan))
+ {
+ case T_ForeignScan:
+ {
+ ForeignScan *fscan = (ForeignScan *) scan;
+
+ context->lockrels = bms_add_members(context->lockrels,
+ fscan->fs_relids);
+ }
+ break;
+
+ case T_SubqueryScan:
+ {
+ SubqueryScan *sscan = (SubqueryScan *) scan;
+
+ (void) ExecGetLockRels((Plan *) sscan->subplan, context);
+ }
+ break;
+
+ case T_CustomScan:
+ {
+ CustomScan *cscan = (CustomScan *) scan;
+ ListCell *lc;
+
+ context->lockrels = bms_add_members(context->lockrels,
+ cscan->custom_relids);
+ foreach(lc, cscan->custom_plans)
+ {
+ (void) ExecGetLockRels((Plan *) lfirst(lc), context);
+ }
+ }
+ break;
+
+ default:
+ context->lockrels = bms_add_member(context->lockrels,
+ scan->scanrelid);
+ break;
+ }
+
+ return true;
+}
/* ----------------------------------------------------------------
* ExecutorStart
@@ -804,6 +983,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
PlannedStmt *plannedstmt = queryDesc->plannedstmt;
+ ExecLockRelsInfo *execlockrelsinfo = queryDesc->execlockrelsinfo;
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
@@ -823,6 +1003,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable);
estate->es_plannedstmt = plannedstmt;
+ estate->es_execlockrelsinfo = execlockrelsinfo;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 5dd8ab7db2..f27f85ab4f 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -66,6 +66,7 @@
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xE000000000000008)
#define PARALLEL_KEY_JIT_INSTRUMENTATION UINT64CONST(0xE000000000000009)
#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xE00000000000000A)
+#define PARALLEL_KEY_EXECLOCKRELSINFO UINT64CONST(0xE00000000000000B)
#define PARALLEL_TUPLE_QUEUE_SIZE 65536
@@ -182,8 +183,10 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt->transientPlan = false;
pstmt->dependsOnRole = false;
pstmt->parallelModeNeeded = false;
+ pstmt->containsInitialPruning = false;
pstmt->planTree = plan;
pstmt->rtable = estate->es_range_table;
+ pstmt->lockrels = NULL;
pstmt->resultRelations = NIL;
pstmt->appendRelations = NIL;
@@ -596,12 +599,15 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
FixedParallelExecutorState *fpes;
char *pstmt_data;
char *pstmt_space;
+ char *execlockrelsinfo_data;
+ char *execlockrelsinfo_space;
char *paramlistinfo_space;
BufferUsage *bufusage_space;
WalUsage *walusage_space;
SharedExecutorInstrumentation *instrumentation = NULL;
SharedJitInstrumentation *jit_instrumentation = NULL;
int pstmt_len;
+ int execlockrelsinfo_len;
int paramlistinfo_len;
int instrumentation_len = 0;
int jit_instrumentation_len = 0;
@@ -630,6 +636,7 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
/* Fix up and serialize plan to be sent to workers. */
pstmt_data = ExecSerializePlan(planstate->plan, estate);
+ execlockrelsinfo_data = nodeToString(estate->es_execlockrelsinfo);
/* Create a parallel context. */
pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers);
@@ -656,6 +663,11 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
shm_toc_estimate_chunk(&pcxt->estimator, pstmt_len);
shm_toc_estimate_keys(&pcxt->estimator, 1);
+ /* Estimate space for serialized ExecLockRelsInfo. */
+ execlockrelsinfo_len = strlen(execlockrelsinfo_data) + 1;
+ shm_toc_estimate_chunk(&pcxt->estimator, execlockrelsinfo_len);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+
/* Estimate space for serialized ParamListInfo. */
paramlistinfo_len = EstimateParamListSpace(estate->es_param_list_info);
shm_toc_estimate_chunk(&pcxt->estimator, paramlistinfo_len);
@@ -750,6 +762,12 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
memcpy(pstmt_space, pstmt_data, pstmt_len);
shm_toc_insert(pcxt->toc, PARALLEL_KEY_PLANNEDSTMT, pstmt_space);
+ /* Store serialized ExecLockRelsInfo */
+ execlockrelsinfo_space = shm_toc_allocate(pcxt->toc, execlockrelsinfo_len);
+ memcpy(execlockrelsinfo_space, execlockrelsinfo_data, execlockrelsinfo_len);
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_EXECLOCKRELSINFO,
+ execlockrelsinfo_space);
+
/* Store serialized ParamListInfo. */
paramlistinfo_space = shm_toc_allocate(pcxt->toc, paramlistinfo_len);
shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARAMLISTINFO, paramlistinfo_space);
@@ -1231,8 +1249,10 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
int instrument_options)
{
char *pstmtspace;
+ char *execlockrelsinfospace;
char *paramspace;
PlannedStmt *pstmt;
+ ExecLockRelsInfo *execlockrelsinfo;
ParamListInfo paramLI;
char *queryString;
@@ -1243,12 +1263,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
pstmtspace = shm_toc_lookup(toc, PARALLEL_KEY_PLANNEDSTMT, false);
pstmt = (PlannedStmt *) stringToNode(pstmtspace);
+ /* Reconstruct leader-supplied ExecLockRelsInfo. */
+ execlockrelsinfospace = shm_toc_lookup(toc, PARALLEL_KEY_EXECLOCKRELSINFO,
+ false);
+ execlockrelsinfo = (ExecLockRelsInfo *) stringToNode(execlockrelsinfospace);
+
/* Reconstruct ParamListInfo. */
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
/* Create a QueryDesc for the query. */
- return CreateQueryDesc(pstmt,
+ return CreateQueryDesc(pstmt, execlockrelsinfo,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 21953f253b..db8c4cd719 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -24,6 +24,7 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "parser/parsetree.h"
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "partitioning/partprune.h"
@@ -183,8 +184,14 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel,
int maxfieldlen);
static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri);
static PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
- PartitionPruneInfo *partitionpruneinfo);
-static Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate);
+ PartitionPruneInfo *partitionpruneinfo,
+ bool consider_initial_steps,
+ bool consider_exec_steps,
+ List *rtable, ExprContext *econtext,
+ PartitionDirectory partdir,
+ Bitmapset **parentrelids);
+static Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate,
+ PartitionPruneInfo *pruneinfo);
static void ExecInitPruningContext(PartitionPruneContext *context,
List *pruning_steps,
PartitionDesc partdesc,
@@ -1483,8 +1490,9 @@ adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri)
* considered to be a stable expression, it can change value from one plan
* node scan to the next during query execution. Stable comparison
* expressions that don't involve such Params allow partition pruning to be
- * done once during executor startup. Expressions that do involve such Params
- * require us to prune separately for each scan of the parent plan node.
+ * done once during executor startup or even before during ExecutorGetLockRels().
+ * Expressions that do involve such Params require us to prune separately for
+ * each scan of the parent plan node.
*
* Note that pruning away unneeded subplans during executor startup has the
* added benefit of not having to initialize the unneeded subplans at all.
@@ -1503,6 +1511,10 @@ adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri)
* updated to account for initial pruning having eliminated some of the
* subplans, if any.
*
+ * ExecGetLockRelsDoInitialPruning:
+ * Do initial pruning as part of ExecGetLockRels() on the parent plan
+ * node
+ *
* ExecFindMatchingSubPlans:
* Returns indexes of matching subplans after evaluating all available
* expressions, that is, using execution pruning steps. This function can
@@ -1531,22 +1543,57 @@ ExecInitPartitionPruning(PlanState *planstate,
{
PartitionPruneState *prunestate;
EState *estate = planstate->state;
+ Plan *plan = planstate->plan;
+ PlanInitPruningOutput *initPruningOutput = NULL;
+ bool do_pruning = (pruneinfo->needs_init_pruning ||
+ pruneinfo->needs_exec_pruning);
- /* We may need an expression context to evaluate partition exprs */
- ExecAssignExprContext(estate, planstate);
+ if (estate->es_execlockrelsinfo)
+ {
+ initPruningOutput = (PlanInitPruningOutput *)
+ ExecFetchPlanInitPruningOutput(estate->es_execlockrelsinfo, plan);
- /*
- * Create the working data structure for pruning.
- */
- prunestate = ExecCreatePartitionPruneState(planstate, pruneinfo);
+ Assert(initPruningOutput != NULL &&
+ IsA(initPruningOutput, PlanInitPruningOutput));
+ /* No need to do initial pruning again, only exec pruning. */
+ do_pruning = pruneinfo->needs_exec_pruning;
+ }
+
+ prunestate = NULL;
+ if (do_pruning)
+ {
+ /* We may need an expression context to evaluate partition exprs */
+ ExecAssignExprContext(estate, planstate);
+
+ /* For data reading, executor always omits detached partitions */
+ if (estate->es_partition_directory == NULL)
+ estate->es_partition_directory =
+ CreatePartitionDirectory(estate->es_query_cxt, false);
+
+ /*
+ * Create the working data structure for pruning. No need to consider
+ * initial pruning steps if we have a PlanInitPruningOutput.
+ */
+ prunestate = ExecCreatePartitionPruneState(planstate, pruneinfo,
+ initPruningOutput == NULL, true,
+ NIL, planstate->ps_ExprContext,
+ estate->es_partition_directory,
+ NULL);
+ }
/*
* Perform an initial partition prune, if required.
*/
- if (prunestate->do_initial_prune)
+ if (initPruningOutput)
+ {
+ /* ExecutorGetLockRels() already did it for us! */
+ *initially_valid_subplans = initPruningOutput->initially_valid_subplans;
+ }
+ else if (prunestate && prunestate->do_initial_prune)
{
/* Determine which subplans survive initial pruning */
- *initially_valid_subplans = ExecFindInitialMatchingSubPlans(prunestate);
+ *initially_valid_subplans = ExecFindInitialMatchingSubPlans(prunestate,
+ pruneinfo);
}
else
{
@@ -1564,7 +1611,7 @@ ExecInitPartitionPruning(PlanState *planstate,
* invalid data in prunestate, because that data won't be consulted again
* (cf initial Assert in ExecFindMatchingSubPlans).
*/
- if (prunestate->do_exec_prune &&
+ if (prunestate && prunestate->do_exec_prune &&
bms_num_members(*initially_valid_subplans) < n_total_subplans)
ExecPartitionPruneFixSubPlanIndexes(prunestate,
*initially_valid_subplans,
@@ -1573,12 +1620,83 @@ ExecInitPartitionPruning(PlanState *planstate,
return prunestate;
}
+/*
+ * ExecGetLockRelsDoInitialPruning
+ * Perform initial pruning as part of doing ExecGetLockRels() on the parent
+ * plan node
+ */
+Bitmapset *
+ExecGetLockRelsDoInitialPruning(Plan *plan, ExecGetLockRelsContext *context,
+ PartitionPruneInfo *pruneinfo)
+{
+ List *rtable = context->stmt->rtable;
+ ParamListInfo params = context->params;
+ ExprContext *econtext;
+ PartitionDirectory pdir;
+ MemoryContext oldcontext,
+ tmpcontext;
+ Bitmapset *parentrelids;
+ PartitionPruneState *prunestate;
+ PlanInitPruningOutput *initPruningOutput;
+
+ /*
+ * A temporary context to allocate stuff needded to run the pruning steps.
+ */
+ tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
+ "initial pruning working data",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(tmpcontext);
+
+ /*
+ * PartitionDirectory to look up partition descriptors, which omits
+ * detached partitions, just like in the executor proper.
+ */
+ pdir = CreatePartitionDirectory(CurrentMemoryContext, false);
+
+ /*
+ * We don't yet have a PlanState for the parent plan node, so must create
+ * a standalone ExprContext to evaluate pruning expressions, equipped with
+ * the information about the EXTERN parameters that the caller passed us.
+ * Note that that's okay because the initial pruning steps do not contain
+ * anything that requires the execution to have started.
+ */
+ econtext = CreateStandaloneExprContext();
+ econtext->ecxt_param_list_info = params;
+ prunestate = ExecCreatePartitionPruneState(NULL, pruneinfo,
+ true, false,
+ rtable, econtext,
+ pdir, &parentrelids);
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Do the pruning and populate a PlanInitPruningOutput for this node. */
+ initPruningOutput = makeNode(PlanInitPruningOutput);
+ initPruningOutput->initially_valid_subplans =
+ ExecFindInitialMatchingSubPlans(prunestate, pruneinfo);
+ ExecStorePlanInitPruningOutput(context, initPruningOutput, plan);
+
+ /*
+ * Report parent partitioned tables as locking targets, though they
+ * would already be locked by ExecCreatePartitionPruneState().
+ */
+ Assert(bms_num_members(parentrelids) > 0);
+ context->lockrels = bms_add_members(context->lockrels, parentrelids);
+
+ FreeExprContext(econtext, true);
+ DestroyPartitionDirectory(pdir);
+ MemoryContextDelete(tmpcontext);
+
+ return initPruningOutput->initially_valid_subplans;
+}
+
/*
* ExecCreatePartitionPruneState
* Build the data structure required for calling
* ExecFindInitialMatchingSubPlans and ExecFindMatchingSubPlans.
*
- * 'planstate' is the parent plan node's execution state.
+ * 'planstate', if not NULL, is the parent plan node's execution state. It
+ * can be NULL if being called before ExecutorStart(), in which case,
+ * 'rtable' (range table), 'econtext', and 'partdir' must be explicitly
+ * provided.
*
* 'partitionpruneinfo' is a PartitionPruneInfo as generated by
* make_partition_pruneinfo. Here we build a PartitionPruneState containing a
@@ -1590,26 +1708,35 @@ ExecInitPartitionPruning(PlanState *planstate,
* as children. The data stored in each PartitionedRelPruningData can be
* re-used each time we re-evaluate which partitions match the pruning steps
* provided in each PartitionedRelPruneInfo.
+ *
+ * The RT indexes of parent partitioned table that are locked here to peruse
+ * their PartitionedRelPruningInfo are returned in *parentrelids if asked
+ * for by the caller.
*/
static PartitionPruneState *
ExecCreatePartitionPruneState(PlanState *planstate,
- PartitionPruneInfo *partitionpruneinfo)
+ PartitionPruneInfo *partitionpruneinfo,
+ bool consider_initial_steps,
+ bool consider_exec_steps,
+ List *rtable, ExprContext *econtext,
+ PartitionDirectory partdir,
+ Bitmapset **parentrelids)
{
- EState *estate = planstate->state;
+ EState *estate = planstate ? planstate->state : NULL;
PartitionPruneState *prunestate;
int n_part_hierarchies;
ListCell *lc;
int i;
- ExprContext *econtext = planstate->ps_ExprContext;
- /* For data reading, executor always omits detached partitions */
- if (estate->es_partition_directory == NULL)
- estate->es_partition_directory =
- CreatePartitionDirectory(estate->es_query_cxt, false);
+ Assert((estate != NULL) ||
+ (partdir != NULL && econtext != NULL && rtable != NIL));
n_part_hierarchies = list_length(partitionpruneinfo->prune_infos);
Assert(n_part_hierarchies > 0);
+ if (parentrelids)
+ *parentrelids = NULL;
+
/*
* Allocate the data structure
*/
@@ -1656,19 +1783,58 @@ ExecCreatePartitionPruneState(PlanState *planstate,
PartitionedRelPruneInfo *pinfo = lfirst_node(PartitionedRelPruneInfo, lc2);
PartitionedRelPruningData *pprune = &prunedata->partrelprunedata[j];
Relation partrel;
+ bool close_partrel = false;
PartitionDesc partdesc;
PartitionKey partkey;
/*
- * We can rely on the copies of the partitioned table's partition
- * key and partition descriptor appearing in its relcache entry,
- * because that entry will be held open and locked for the
- * duration of this executor run.
+ * Must open the relation by ourselves when called before the
+ * execution has started, such as, when called during
+ * ExecutorGetLockRels() on a cached plan. In that case,
+ * sub-partitions must be locked, because AcquirePlannerLocks()
+ * would not have seen them. (1st relation in a partrelpruneinfos
+ * list is always the root partitioned table appearing in the
+ * query, which AcquirePlannerLocks() would have locked; the
+ * Assert in relation_open() guards that assumption.)
+ */
+ if (estate == NULL)
+ {
+ RangeTblEntry *rte = rt_fetch(pinfo->rtindex, rtable);
+ int lockmode = (j == 0) ? NoLock : rte->rellockmode;
+
+ partrel = table_open(rte->relid, lockmode);
+ close_partrel = true;
+
+ /*
+ * Also report the partitioned table as having been locked.
+ * XXX - actually, *parentrelids set is later merged by the
+ * caller into the set of relations "to-be locked" by
+ * AcquireExecutorLocks(), thus causing the lock on this
+ * table to be requested again.
+ */
+ Assert(parentrelids != NULL);
+ *parentrelids = bms_add_member(*parentrelids, pinfo->rtindex);
+ }
+ else
+ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+
+ /*
+ * We can rely on the copy of the partitioned table's partition
+ * key from in its relcache entry, because it can't change (or
+ * get destroyed) as long as the relation is locked. Partition
+ * descriptor is taken from the PartitionDirectory associated with
+ * the table that is held open long enough for the descriptor to
+ * remain valid while it's used to perform the pruning steps.
*/
- partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
partkey = RelationGetPartitionKey(partrel);
- partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
- partrel);
+ partdesc = PartitionDirectoryLookup(partdir, partrel);
+
+ /*
+ * Must close partrel, keeping the lock taken, if we're not using
+ * EState's entry.
+ */
+ if (close_partrel)
+ table_close(partrel, NoLock);
/*
* Initialize the subplan_map and subpart_map.
@@ -1770,7 +1936,7 @@ ExecCreatePartitionPruneState(PlanState *planstate,
* Initialize pruning contexts as needed.
*/
pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
- if (pinfo->initial_pruning_steps)
+ if (consider_initial_steps && pinfo->initial_pruning_steps)
{
ExecInitPruningContext(&pprune->initial_context,
pinfo->initial_pruning_steps,
@@ -1780,7 +1946,7 @@ ExecCreatePartitionPruneState(PlanState *planstate,
prunestate->do_initial_prune = true;
}
pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
- if (pinfo->exec_pruning_steps)
+ if (consider_exec_steps && pinfo->exec_pruning_steps)
{
ExecInitPruningContext(&pprune->exec_context,
pinfo->exec_pruning_steps,
@@ -1899,7 +2065,8 @@ ExecInitPruningContext(PartitionPruneContext *context,
* is required.
*/
static Bitmapset *
-ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate)
+ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate,
+ PartitionPruneInfo *pruneinfo)
{
Bitmapset *result = NULL;
MemoryContext oldcontext;
@@ -1909,8 +2076,8 @@ ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate)
Assert(prunestate->do_initial_prune);
/*
- * Switch to a temp context to avoid leaking memory in the executor's
- * query-lifespan memory context.
+ * Switch to a temp context to avoid leaking memory in the longer-term
+ * memory context.
*/
oldcontext = MemoryContextSwitchTo(prunestate->prune_context);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 9df1f81ea8..7246f9175f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -119,6 +119,7 @@ CreateExecutorState(void)
estate->es_relations = NULL;
estate->es_rowmarks = NULL;
estate->es_plannedstmt = NULL;
+ estate->es_execlockrelsinfo = NULL;
estate->es_junkFilter = NULL;
@@ -785,6 +786,13 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rti > 0 && rti <= estate->es_range_table_size);
+ /*
+ * A cross-check that AcquireExecutorLocks() hasn't missed any relations
+ * it must not have.
+ */
+ Assert(estate->es_execlockrelsinfo == NULL ||
+ bms_is_member(rti, estate->es_execlockrelsinfo->lockrels));
+
rel = estate->es_relations[rti - 1];
if (rel == NULL)
{
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f9460ae506..a2182a6b1f 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -844,7 +844,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
else
dest = None_Receiver;
- es->qd = CreateQueryDesc(es->stmt,
+ es->qd = CreateQueryDesc(es->stmt, NULL,
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 5b6d3eb23b..966615f670 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -94,6 +94,45 @@ static bool ExecAppendAsyncRequest(AppendState *node, TupleTableSlot **result);
static void ExecAppendAsyncEventWait(AppendState *node);
static void classify_matching_subplans(AppendState *node);
+/* ----------------------------------------------------------------
+ * ExecGetAppendLockRels
+ * Do ExecGetLockRels()'s work for an Append plan
+ * ----------------------------------------------------------------
+ */
+bool
+ExecGetAppendLockRels(Append *node, ExecGetLockRelsContext *context)
+{
+ PartitionPruneInfo *pruneinfo = node->part_prune_info;
+
+ if (pruneinfo && pruneinfo->needs_init_pruning)
+ {
+ List *subplans = node->appendplans;
+ Bitmapset *validsubplans;
+ int i;
+
+ validsubplans = ExecGetLockRelsDoInitialPruning((Plan *) node,
+ context, pruneinfo);
+
+ /* Prep the surviving subplans. */
+ i = -1;
+ while ((i = bms_next_member(validsubplans, i)) >= 0)
+ {
+ Plan *subplan = list_nth(subplans, i);
+
+ (void) ExecGetLockRels(subplan, context);
+ }
+
+ /* done with this node */
+ return true;
+ }
+
+ /*
+ * Look at all subplans, which the caller would do by calling
+ * plan_tree_walker() on the node.
+ */
+ return false;
+}
+
/* ----------------------------------------------------------------
* ExecInitAppend
*
@@ -155,7 +194,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
* subplan, we can fill as_valid_subplans immediately, preventing
* later calls to ExecFindMatchingSubPlans.
*/
- if (!prunestate->do_exec_prune && nplans > 0)
+ if (appendstate->as_prune_state == NULL ||
+ (!appendstate->as_prune_state->do_exec_prune && nplans > 0))
appendstate->as_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
}
else
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 9a9f29e845..869b836a14 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -54,6 +54,45 @@ typedef int32 SlotNumber;
static TupleTableSlot *ExecMergeAppend(PlanState *pstate);
static int heap_compare_slots(Datum a, Datum b, void *arg);
+/* ----------------------------------------------------------------
+ * ExecGetMergeAppendLockRels
+ * Do ExecGetLockRels()'s work for a MergeAppend plan
+ * ----------------------------------------------------------------
+ */
+bool
+ExecGetMergeAppendLockRels(MergeAppend *node, ExecGetLockRelsContext *context)
+{
+ PartitionPruneInfo *pruneinfo = node->part_prune_info;
+
+ if (pruneinfo && pruneinfo->needs_init_pruning)
+ {
+ List *subplans = node->mergeplans;
+ Bitmapset *validsubplans;
+ int i;
+
+ validsubplans = ExecGetLockRelsDoInitialPruning((Plan *) node,
+ context, pruneinfo);
+
+ /* Prep the surviving subplans. */
+ i = -1;
+ while ((i = bms_next_member(validsubplans, i)) >= 0)
+ {
+ Plan *subplan = list_nth(subplans, i);
+
+ (void) ExecGetLockRels(subplan, context);
+ }
+
+ /* done with this node */
+ return true;
+ }
+
+ /*
+ * Look at all subplans, which the caller would do by calling
+ * plan_tree_walker() on the node.
+ */
+ return false;
+}
+
/* ----------------------------------------------------------------
* ExecInitMergeAppend
@@ -103,7 +142,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
* subplan, we can fill as_valid_subplans immediately, preventing
* later calls to ExecFindMatchingSubPlans.
*/
- if (!prunestate->do_exec_prune && nplans > 0)
+ if (mergestate->ms_prune_state == NULL ||
+ (!mergestate->ms_prune_state->do_exec_prune && nplans > 0))
mergestate->ms_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
}
else
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5ec699a9bd..c860045fcb 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2700,6 +2700,30 @@ ExecLookupResultRelByOid(ModifyTableState *node, Oid resultoid,
return NULL;
}
+/*
+ * ExecGetModifyTableLockRels
+ * Do ExecGetLockRels()'s work for a ModifyTable plan
+ */
+bool
+ExecGetModifyTableLockRels(ModifyTable *plan, ExecGetLockRelsContext *context)
+{
+ ListCell *lc;
+
+ if (plan->rootRelation > 0)
+ context->lockrels = bms_add_member(context->lockrels,
+ plan->rootRelation);
+ context->lockrels = bms_add_member(context->lockrels,
+ plan->nominalRelation);
+ foreach(lc, plan->resultRelations)
+ {
+ context->lockrels = bms_add_member(context->lockrels,
+ lfirst_int(lc));
+ }
+
+ /* caller will look at the source subplan */
+ return false;
+}
+
/* ----------------------------------------------------------------
* ExecInitModifyTable
* ----------------------------------------------------------------
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index a82e986667..2107009591 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1578,6 +1578,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
CachedPlanSource *plansource;
CachedPlan *cplan;
List *stmt_list;
+ List *execlockrelsinfo_list;
char *query_string;
Snapshot snapshot;
MemoryContext oldcontext;
@@ -1659,6 +1660,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
/* Replan if needed, and increment plan refcount for portal */
cplan = GetCachedPlan(plansource, paramLI, NULL, _SPI_current->queryEnv);
stmt_list = cplan->stmt_list;
+ execlockrelsinfo_list = cplan->execlockrelsinfo_list;
if (!plan->saved)
{
@@ -1670,6 +1672,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
*/
oldcontext = MemoryContextSwitchTo(portal->portalContext);
stmt_list = copyObject(stmt_list);
+ execlockrelsinfo_list = copyObject(execlockrelsinfo_list);
MemoryContextSwitchTo(oldcontext);
ReleaseCachedPlan(cplan, NULL);
cplan = NULL; /* portal shouldn't depend on cplan */
@@ -1683,6 +1686,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
query_string,
plansource->commandTag,
stmt_list,
+ execlockrelsinfo_list,
cplan);
/*
@@ -2473,7 +2477,9 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
CachedPlanSource *plansource = (CachedPlanSource *) lfirst(lc1);
List *stmt_list;
- ListCell *lc2;
+ List *execlockrelsinfo_list;
+ ListCell *lc2,
+ *lc3;
spicallbackarg.query = plansource->query_string;
@@ -2552,6 +2558,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
plan_owner, _SPI_current->queryEnv);
stmt_list = cplan->stmt_list;
+ execlockrelsinfo_list = cplan->execlockrelsinfo_list;
/*
* If we weren't given a specific snapshot to use, and the statement
@@ -2589,9 +2596,10 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
}
}
- foreach(lc2, stmt_list)
+ forboth(lc2, stmt_list, lc3, execlockrelsinfo_list)
{
PlannedStmt *stmt = lfirst_node(PlannedStmt, lc2);
+ ExecLockRelsInfo *execlockrelsinfo = lfirst_node(ExecLockRelsInfo, lc3);
bool canSetTag = stmt->canSetTag;
DestReceiver *dest;
@@ -2663,7 +2671,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
else
snap = InvalidSnapshot;
- qdesc = CreateQueryDesc(stmt,
+ qdesc = CreateQueryDesc(stmt, execlockrelsinfo,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d4f8455a2b..68c664070c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -68,6 +68,13 @@
} \
} while (0)
+/* Copy a field that is an array with numElem ints */
+#define COPY_INT_ARRAY(fldname, numElem) \
+ do { \
+ newnode->fldname = (numElem) > 0 ? palloc((numElem) * sizeof(int)) : NULL; \
+ memcpy(newnode->fldname, from->fldname, sizeof(int) * (numElem)); \
+ } while (0)
+
/* Copy a parse location field (for Copy, this is same as scalar case) */
#define COPY_LOCATION_FIELD(fldname) \
(newnode->fldname = from->fldname)
@@ -94,9 +101,12 @@ _copyPlannedStmt(const PlannedStmt *from)
COPY_SCALAR_FIELD(transientPlan);
COPY_SCALAR_FIELD(dependsOnRole);
COPY_SCALAR_FIELD(parallelModeNeeded);
+ COPY_SCALAR_FIELD(containsInitialPruning);
COPY_SCALAR_FIELD(jitFlags);
COPY_NODE_FIELD(planTree);
+ COPY_SCALAR_FIELD(numPlanNodes);
COPY_NODE_FIELD(rtable);
+ COPY_BITMAPSET_FIELD(lockrels);
COPY_NODE_FIELD(resultRelations);
COPY_NODE_FIELD(appendRelations);
COPY_NODE_FIELD(subplans);
@@ -1278,6 +1288,8 @@ _copyPartitionPruneInfo(const PartitionPruneInfo *from)
PartitionPruneInfo *newnode = makeNode(PartitionPruneInfo);
COPY_NODE_FIELD(prune_infos);
+ COPY_SCALAR_FIELD(needs_init_pruning);
+ COPY_SCALAR_FIELD(needs_exec_pruning);
COPY_BITMAPSET_FIELD(other_subplans);
return newnode;
@@ -4941,6 +4953,33 @@ _copyExtensibleNode(const ExtensibleNode *from)
return newnode;
}
+/* ****************************************************************
+ * execnodes.h copy functions
+ * ****************************************************************
+ */
+static ExecLockRelsInfo *
+_copyExecLockRelsInfo(const ExecLockRelsInfo *from)
+{
+ ExecLockRelsInfo *newnode = makeNode(ExecLockRelsInfo);
+
+ COPY_BITMAPSET_FIELD(lockrels);
+ COPY_SCALAR_FIELD(numPlanNodes);
+ COPY_NODE_FIELD(initPruningOutputs);
+ COPY_INT_ARRAY(ipoIndexes, from->numPlanNodes);
+
+ return newnode;
+}
+
+static PlanInitPruningOutput *
+_copyPlanInitPruningOutput(const PlanInitPruningOutput *from)
+{
+ PlanInitPruningOutput *newnode = makeNode(PlanInitPruningOutput);
+
+ COPY_BITMAPSET_FIELD(initially_valid_subplans);
+
+ return newnode;
+}
+
/* ****************************************************************
* value.h copy functions
* ****************************************************************
@@ -4995,7 +5034,6 @@ _copyBitString(const BitString *from)
return newnode;
}
-
static ForeignKeyCacheInfo *
_copyForeignKeyCacheInfo(const ForeignKeyCacheInfo *from)
{
@@ -5944,6 +5982,16 @@ copyObjectImpl(const void *from)
retval = _copyPublicationTable(from);
break;
+ /*
+ * EXECUTION NODES
+ */
+ case T_ExecLockRelsInfo:
+ retval = _copyExecLockRelsInfo(from);
+ break;
+ case T_PlanInitPruningOutput:
+ retval = _copyPlanInitPruningOutput(from);
+ break;
+
/*
* MISCELLANEOUS NODES
*/
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6bdad462c7..e0e09d7abd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -312,9 +312,12 @@ _outPlannedStmt(StringInfo str, const PlannedStmt *node)
WRITE_BOOL_FIELD(transientPlan);
WRITE_BOOL_FIELD(dependsOnRole);
WRITE_BOOL_FIELD(parallelModeNeeded);
+ WRITE_BOOL_FIELD(containsInitialPruning);
WRITE_INT_FIELD(jitFlags);
WRITE_NODE_FIELD(planTree);
+ WRITE_INT_FIELD(numPlanNodes);
WRITE_NODE_FIELD(rtable);
+ WRITE_BITMAPSET_FIELD(lockrels);
WRITE_NODE_FIELD(resultRelations);
WRITE_NODE_FIELD(appendRelations);
WRITE_NODE_FIELD(subplans);
@@ -1004,6 +1007,8 @@ _outPartitionPruneInfo(StringInfo str, const PartitionPruneInfo *node)
WRITE_NODE_TYPE("PARTITIONPRUNEINFO");
WRITE_NODE_FIELD(prune_infos);
+ WRITE_BOOL_FIELD(needs_init_pruning);
+ WRITE_BOOL_FIELD(needs_exec_pruning);
WRITE_BITMAPSET_FIELD(other_subplans);
}
@@ -2274,6 +2279,7 @@ _outPlannerGlobal(StringInfo str, const PlannerGlobal *node)
WRITE_NODE_FIELD(subplans);
WRITE_BITMAPSET_FIELD(rewindPlanIDs);
WRITE_NODE_FIELD(finalrtable);
+ WRITE_BITMAPSET_FIELD(lockrels);
WRITE_NODE_FIELD(finalrowmarks);
WRITE_NODE_FIELD(resultRelations);
WRITE_NODE_FIELD(appendRelations);
@@ -2697,6 +2703,31 @@ _outExtensibleNode(StringInfo str, const ExtensibleNode *node)
methods->nodeOut(str, node);
}
+/*****************************************************************************
+ *
+ * Stuff from execnodes.h
+ *
+ *****************************************************************************/
+
+static void
+_outExecLockRelsInfo(StringInfo str, const ExecLockRelsInfo *node)
+{
+ WRITE_NODE_TYPE("EXECLOCKRELSINFO");
+
+ WRITE_BITMAPSET_FIELD(lockrels);
+ WRITE_INT_FIELD(numPlanNodes);
+ WRITE_NODE_FIELD(initPruningOutputs);
+ WRITE_INT_ARRAY(ipoIndexes, node->numPlanNodes);
+}
+
+static void
+_outPlanInitPruningOutput(StringInfo str, const PlanInitPruningOutput *node)
+{
+ WRITE_NODE_TYPE("PARTITIONINITPRUNINGOUTPUT");
+
+ WRITE_BITMAPSET_FIELD(initially_valid_subplans);
+}
+
/*****************************************************************************
*
* Stuff from parsenodes.h.
@@ -4538,6 +4569,16 @@ outNode(StringInfo str, const void *obj)
_outPartitionRangeDatum(str, obj);
break;
+ /*
+ * EXECUTION NODES
+ */
+ case T_ExecLockRelsInfo:
+ _outExecLockRelsInfo(str, obj);
+ break;
+ case T_PlanInitPruningOutput:
+ _outPlanInitPruningOutput(str, obj);
+ break;
+
default:
/*
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3f68f7c18d..41ded72c4c 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1585,9 +1585,12 @@ _readPlannedStmt(void)
READ_BOOL_FIELD(transientPlan);
READ_BOOL_FIELD(dependsOnRole);
READ_BOOL_FIELD(parallelModeNeeded);
+ READ_BOOL_FIELD(containsInitialPruning);
READ_INT_FIELD(jitFlags);
READ_NODE_FIELD(planTree);
+ READ_INT_FIELD(numPlanNodes);
READ_NODE_FIELD(rtable);
+ READ_BITMAPSET_FIELD(lockrels);
READ_NODE_FIELD(resultRelations);
READ_NODE_FIELD(appendRelations);
READ_NODE_FIELD(subplans);
@@ -2534,6 +2537,8 @@ _readPartitionPruneInfo(void)
READ_LOCALS(PartitionPruneInfo);
READ_NODE_FIELD(prune_infos);
+ READ_BOOL_FIELD(needs_init_pruning);
+ READ_BOOL_FIELD(needs_exec_pruning);
READ_BITMAPSET_FIELD(other_subplans);
READ_DONE();
@@ -2703,6 +2708,35 @@ _readPartitionRangeDatum(void)
READ_DONE();
}
+/*
+ * _readExecLockRelsInfo
+ */
+static ExecLockRelsInfo *
+_readExecLockRelsInfo(void)
+{
+ READ_LOCALS(ExecLockRelsInfo);
+
+ READ_BITMAPSET_FIELD(lockrels);
+ READ_INT_FIELD(numPlanNodes);
+ READ_NODE_FIELD(initPruningOutputs);
+ READ_INT_ARRAY(ipoIndexes, local_node->numPlanNodes);
+
+ READ_DONE();
+}
+
+/*
+ * _readPlanInitPruningOutput
+ */
+static PlanInitPruningOutput *
+_readPlanInitPruningOutput(void)
+{
+ READ_LOCALS(PlanInitPruningOutput);
+
+ READ_BITMAPSET_FIELD(initially_valid_subplans);
+
+ READ_DONE();
+}
+
/*
* parseNodeString
*
@@ -2974,6 +3008,10 @@ parseNodeString(void)
return_value = _readPartitionBoundSpec();
else if (MATCH("PARTITIONRANGEDATUM", 19))
return_value = _readPartitionRangeDatum();
+ else if (MATCH("EXECLOCKRELSINFO", 16))
+ return_value = _readExecLockRelsInfo();
+ else if (MATCH("PARTITIONINITPRUNINGOUTPUT", 26))
+ return_value = _readPlanInitPruningOutput();
else
{
elog(ERROR, "badly formatted node string \"%.32s\"...", token);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bd09f85aea..9e41bbd228 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -517,8 +517,11 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->transientPlan = glob->transientPlan;
result->dependsOnRole = glob->dependsOnRole;
result->parallelModeNeeded = glob->parallelModeNeeded;
+ result->containsInitialPruning = glob->containsInitialPruning;
result->planTree = top_plan;
+ result->numPlanNodes = glob->lastPlanNodeId;
result->rtable = glob->finalrtable;
+ result->lockrels = glob->lockrels;
result->resultRelations = glob->resultRelations;
result->appendRelations = glob->appendRelations;
result->subplans = glob->subplans;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index a7b11b7f03..cee8c570fd 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -483,6 +483,7 @@ static void
add_rte_to_flat_rtable(PlannerGlobal *glob, RangeTblEntry *rte)
{
RangeTblEntry *newrte;
+ Index rti = list_length(glob->finalrtable) + 1;
/* flat copy to duplicate all the scalar fields */
newrte = (RangeTblEntry *) palloc(sizeof(RangeTblEntry));
@@ -517,7 +518,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, RangeTblEntry *rte)
* but it would probably cost more cycles than it would save.
*/
if (newrte->rtekind == RTE_RELATION)
+ {
+ glob->lockrels = bms_add_member(glob->lockrels, rti);
glob->relationOids = lappend_oid(glob->relationOids, newrte->relid);
+ }
}
/*
@@ -1548,6 +1552,9 @@ set_append_references(PlannerInfo *root,
pinfo->rtindex += rtoffset;
}
}
+
+ if (aplan->part_prune_info->needs_init_pruning)
+ root->glob->containsInitialPruning = true;
}
/* We don't need to recurse to lefttree or righttree ... */
@@ -1620,6 +1627,9 @@ set_mergeappend_references(PlannerInfo *root,
pinfo->rtindex += rtoffset;
}
}
+
+ if (mplan->part_prune_info->needs_init_pruning)
+ root->glob->containsInitialPruning = true;
}
/* We don't need to recurse to lefttree or righttree ... */
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7080cb25d9..3322dc79f2 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -144,7 +144,9 @@ static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
List *prunequal,
Bitmapset *partrelids,
int *relid_subplan_map,
- Bitmapset **matchedsubplans);
+ Bitmapset **matchedsubplans,
+ bool *needs_init_pruning,
+ bool *needs_exec_pruning);
static void gen_partprune_steps(RelOptInfo *rel, List *clauses,
PartClauseTarget target,
GeneratePruningStepsContext *context);
@@ -230,6 +232,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
int *relid_subplan_map;
ListCell *lc;
int i;
+ bool needs_init_pruning = false;
+ bool needs_exec_pruning = false;
/*
* Scan the subpaths to see which ones are scans of partition child
@@ -309,12 +313,16 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Bitmapset *partrelids = (Bitmapset *) lfirst(lc);
List *pinfolist;
Bitmapset *matchedsubplans = NULL;
+ bool partrel_needs_init_pruning;
+ bool partrel_needs_exec_pruning;
pinfolist = make_partitionedrel_pruneinfo(root, parentrel,
prunequal,
partrelids,
relid_subplan_map,
- &matchedsubplans);
+ &matchedsubplans,
+ &partrel_needs_init_pruning,
+ &partrel_needs_exec_pruning);
/* When pruning is possible, record the matched subplans */
if (pinfolist != NIL)
@@ -323,6 +331,10 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
allmatchedsubplans = bms_join(matchedsubplans,
allmatchedsubplans);
}
+ if (!needs_init_pruning)
+ needs_init_pruning = partrel_needs_init_pruning;
+ if (!needs_exec_pruning)
+ needs_exec_pruning = partrel_needs_exec_pruning;
}
pfree(relid_subplan_map);
@@ -337,6 +349,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
/* Else build the result data structure */
pruneinfo = makeNode(PartitionPruneInfo);
pruneinfo->prune_infos = prunerelinfos;
+ pruneinfo->needs_init_pruning = needs_init_pruning;
+ pruneinfo->needs_exec_pruning = needs_exec_pruning;
/*
* Some subplans may not belong to any of the identified partitioned rels.
@@ -435,13 +449,18 @@ add_part_relids(List *allpartrelids, Bitmapset *partrelids)
* If we cannot find any useful run-time pruning steps, return NIL.
* However, on success, each rel identified in partrelids will have
* an element in the result list, even if some of them are useless.
+ * *needs_init_pruning and *needs_exec_pruning are set to indicate that the
+ * returned PartitionedRelPruneInfos contains pruning steps that can be
+ * performed before and after execution begins, respectively.
*/
static List *
make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *prunequal,
Bitmapset *partrelids,
int *relid_subplan_map,
- Bitmapset **matchedsubplans)
+ Bitmapset **matchedsubplans,
+ bool *needs_init_pruning,
+ bool *needs_exec_pruning)
{
RelOptInfo *targetpart = NULL;
List *pinfolist = NIL;
@@ -452,6 +471,10 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
int rti;
int i;
+ /* Will find out below. */
+ *needs_init_pruning = false;
+ *needs_exec_pruning = false;
+
/*
* Examine each partitioned rel, constructing a temporary array to map
* from planner relids to index of the partitioned rel, and building a
@@ -539,6 +562,9 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* executor per-scan pruning steps. This first pass creates startup
* pruning steps and detects whether there's any possibly-useful quals
* that would require per-scan pruning.
+ *
+ * In the first pass, we note whether the 2nd pass is necessary by
+ * by noting the presence of EXEC parameters.
*/
gen_partprune_steps(subpart, partprunequal, PARTTARGET_INITIAL,
&context);
@@ -613,6 +639,11 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
pinfo->execparamids = execparamids;
/* Remaining fields will be filled in the next loop */
+ if (!*needs_init_pruning)
+ *needs_init_pruning = (initial_pruning_steps != NIL);
+ if (!*needs_exec_pruning)
+ *needs_exec_pruning = (exec_pruning_steps != NIL);
+
pinfolist = lappend(pinfolist, pinfo);
}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index ba2fcfeb4a..085eb3f209 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -945,15 +945,17 @@ pg_plan_query(Query *querytree, const char *query_string, int cursorOptions,
* For normal optimizable statements, invoke the planner. For utility
* statements, just make a wrapper PlannedStmt node.
*
- * The result is a list of PlannedStmt nodes.
+ * The result is a list of PlannedStmt nodes. Also, a NULL is appended to
+ * *execlockrelsinfo_list for each PlannedStmt added to the returned list.
*/
List *
pg_plan_queries(List *querytrees, const char *query_string, int cursorOptions,
- ParamListInfo boundParams)
+ ParamListInfo boundParams, List **execlockrelsinfo_list)
{
List *stmt_list = NIL;
ListCell *query_list;
+ *execlockrelsinfo_list = NIL;
foreach(query_list, querytrees)
{
Query *query = lfirst_node(Query, query_list);
@@ -977,6 +979,7 @@ pg_plan_queries(List *querytrees, const char *query_string, int cursorOptions,
}
stmt_list = lappend(stmt_list, stmt);
+ *execlockrelsinfo_list = lappend(*execlockrelsinfo_list, NULL);
}
return stmt_list;
@@ -1080,7 +1083,8 @@ exec_simple_query(const char *query_string)
QueryCompletion qc;
MemoryContext per_parsetree_context = NULL;
List *querytree_list,
- *plantree_list;
+ *plantree_list,
+ *plantree_execlockrelsinfo_list;
Portal portal;
DestReceiver *receiver;
int16 format;
@@ -1167,7 +1171,8 @@ exec_simple_query(const char *query_string)
NULL, 0, NULL);
plantree_list = pg_plan_queries(querytree_list, query_string,
- CURSOR_OPT_PARALLEL_OK, NULL);
+ CURSOR_OPT_PARALLEL_OK, NULL,
+ &plantree_execlockrelsinfo_list);
/*
* Done with the snapshot used for parsing/planning.
@@ -1203,6 +1208,7 @@ exec_simple_query(const char *query_string)
query_string,
commandTag,
plantree_list,
+ plantree_execlockrelsinfo_list,
NULL);
/*
@@ -1991,6 +1997,7 @@ exec_bind_message(StringInfo input_message)
query_string,
psrc->commandTag,
cplan->stmt_list,
+ cplan->execlockrelsinfo_list,
cplan);
/* Done with the snapshot used for parameter I/O and parsing/planning */
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5f907831a3..972ddc014e 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -35,7 +35,7 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
+static void ProcessQuery(PlannedStmt *plan, ExecLockRelsInfo *execlockrelsinfo,
const char *sourceText,
ParamListInfo params,
QueryEnvironment *queryEnv,
@@ -65,6 +65,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ ExecLockRelsInfo *execlockrelsinfo,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +78,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->execlockrelsinfo = execlockrelsinfo; /* ExecutorGetLockRels() output for plan */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -122,6 +124,7 @@ FreeQueryDesc(QueryDesc *qdesc)
* PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
*
* plan: the plan tree for the query
+ * execlockrelsinfo: ExecutorGetLockRels() output for the plan tree
* sourceText: the source text of the query
* params: any parameters needed
* dest: where to send results
@@ -134,6 +137,7 @@ FreeQueryDesc(QueryDesc *qdesc)
*/
static void
ProcessQuery(PlannedStmt *plan,
+ ExecLockRelsInfo *execlockrelsinfo,
const char *sourceText,
ParamListInfo params,
QueryEnvironment *queryEnv,
@@ -145,7 +149,7 @@ ProcessQuery(PlannedStmt *plan,
/*
* Create the QueryDesc object
*/
- queryDesc = CreateQueryDesc(plan, sourceText,
+ queryDesc = CreateQueryDesc(plan, execlockrelsinfo, sourceText,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
@@ -490,6 +494,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ linitial_node(ExecLockRelsInfo, portal->execlockrelsinfos),
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -1190,7 +1195,8 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *stmtlist_item,
+ *execlockrelsinfolist_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1211,9 +1217,12 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ forboth(stmtlist_item, portal->stmts,
+ execlockrelsinfolist_item, portal->execlockrelsinfos)
{
PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ ExecLockRelsInfo *execlockrelsinfo = lfirst_node(ExecLockRelsInfo,
+ execlockrelsinfolist_item);
/*
* If we got a cancel signal in prior command, quit
@@ -1271,7 +1280,7 @@ PortalRunMulti(Portal portal,
if (pstmt->canSetTag)
{
/* statement can set tag string */
- ProcessQuery(pstmt,
+ ProcessQuery(pstmt, execlockrelsinfo,
portal->sourceText,
portal->portalParams,
portal->queryEnv,
@@ -1280,7 +1289,7 @@ PortalRunMulti(Portal portal,
else
{
/* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
+ ProcessQuery(pstmt, execlockrelsinfo,
portal->sourceText,
portal->portalParams,
portal->queryEnv,
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 4cf6db504f..c40a6f19d6 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -99,14 +99,15 @@ static dlist_head cached_expression_list = DLIST_STATIC_INIT(cached_expression_l
static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
-static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool CheckCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
+static List *AcquireExecutorLocks(List *stmt_list, ParamListInfo boundParams);
+static void ReleaseExecutorLocks(List *stmt_list, List *execlockrelsinfo_list);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -782,6 +783,47 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
return tlist;
}
+/*
+ * CachedPlanSaveExecLockRelsInfos
+ * Save the list containing ExecLockRelsInfo nodes in the given CachedPlan
+ *
+ * The provided list is copied into a dedicated context that is a child of
+ * plan->context.
+ */
+static void
+CachedPlanSaveExecLockRelsInfos(CachedPlan *plan, List *execlockrelsinfo_list)
+{
+ MemoryContext execlockrelsinfo_context = plan->execlockrelsinfo_context,
+ oldcontext = CurrentMemoryContext;
+ List *execlockrelsinfo_list_copy;
+
+ /*
+ * Set up the dedicated context if not already done, saving it as a child
+ * of the CachedPlan's context.
+ */
+ if (execlockrelsinfo_context == NULL)
+ {
+ execlockrelsinfo_context = AllocSetContextCreate(CurrentMemoryContext,
+ "CachedPlan execlockrelsinfo list",
+ ALLOCSET_START_SMALL_SIZES);
+ MemoryContextSetParent(execlockrelsinfo_context, plan->context);
+ MemoryContextSetIdentifier(execlockrelsinfo_context, plan->context->ident);
+ plan->execlockrelsinfo_context = execlockrelsinfo_context;
+ }
+ else
+ {
+ /* Just clear existing contents by resetting the context. */
+ Assert(MemoryContextIsValid(execlockrelsinfo_context));
+ MemoryContextReset(execlockrelsinfo_context);
+ }
+
+ MemoryContextSwitchTo(execlockrelsinfo_context);
+ execlockrelsinfo_list_copy = copyObject(execlockrelsinfo_list);
+ MemoryContextSwitchTo(oldcontext);
+
+ plan->execlockrelsinfo_list = execlockrelsinfo_list_copy;
+}
+
/*
* CheckCachedPlan: see if the CachedPlanSource's generic plan is valid.
*
@@ -790,9 +832,17 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
*
* On a "true" return, we have acquired the locks needed to run the plan.
* (We must do this for the "true" result to be race-condition-free.)
+ *
+ * If the CachedPlan is valid, this calls ExecutorGetLockRels on each
+ * PlannedStmt contained in it to determine the set of relations to lock by
+ * AcquireExecutorLocks(). Resulting ExecLockRelsInfo nodes, allocated in a
+ * child context of the context containing the plan itself, are added into
+ * plan->execlockrelsinfo_list. ExecLockRelsInfo nodes that may be present
+ * in the list from the last invocation of CheckCachedPlan() on the same
+ * CachedPlan are deleted.
*/
static bool
-CheckCachedPlan(CachedPlanSource *plansource)
+CheckCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams)
{
CachedPlan *plan = plansource->gplan;
@@ -820,13 +870,22 @@ CheckCachedPlan(CachedPlanSource *plansource)
*/
if (plan->is_valid)
{
+ List *execlockrelsinfo_list;
+
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
Assert(plan->refcount > 0);
- AcquireExecutorLocks(plan->stmt_list, true);
+ /*
+ * Lock relations scanned by the plan. This also invokes
+ * ExecutorGetLockRels() to do initial partition pruning on the plan
+ * tree iff some nodes in it are marked as needing it. Relations whose
+ * scan nodes are pruned as a result of that are not locked here.
+ */
+ execlockrelsinfo_list = AcquireExecutorLocks(plan->stmt_list,
+ boundParams);
/*
* If plan was transient, check to see if TransactionXmin has
@@ -844,11 +903,14 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (plan->is_valid)
{
/* Successfully revalidated and locked the query. */
+
+ /* Remember ExecLockRelsInfos in the CachedPlan. */
+ CachedPlanSaveExecLockRelsInfos(plan, execlockrelsinfo_list);
return true;
}
/* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ ReleaseExecutorLocks(plan->stmt_list, execlockrelsinfo_list);
}
/*
@@ -880,7 +942,8 @@ BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv)
{
CachedPlan *plan;
- List *plist;
+ List *plist,
+ *execlockrelsinfo_list;
bool snapshot_set;
bool is_transient;
MemoryContext plan_context;
@@ -933,7 +996,8 @@ BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
* Generate the plan.
*/
plist = pg_plan_queries(qlist, plansource->query_string,
- plansource->cursor_options, boundParams);
+ plansource->cursor_options, boundParams,
+ &execlockrelsinfo_list);
/* Release snapshot if we got one */
if (snapshot_set)
@@ -1002,6 +1066,11 @@ BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
plan->is_saved = false;
plan->is_valid = true;
+ /* Save the dummy ExecLockRelsInfo list. */
+ plan->execlockrelsinfo_context = NULL;
+ CachedPlanSaveExecLockRelsInfos(plan, execlockrelsinfo_list);
+ Assert(MemoryContextIsValid(plan->execlockrelsinfo_context));
+
/* assign generation number to new plan */
plan->generation = ++(plansource->generation);
@@ -1160,7 +1229,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
if (!customplan)
{
- if (CheckCachedPlan(plansource))
+ if (CheckCachedPlan(plansource, boundParams))
{
/* We want a generic plan, and we already have a valid one */
plan = plansource->gplan;
@@ -1366,7 +1435,6 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
foreach(lc, plan->stmt_list)
{
PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc);
- ListCell *lc2;
if (plannedstmt->commandType == CMD_UTILITY)
return false;
@@ -1375,13 +1443,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
* We have to grovel through the rtable because it's likely to contain
* an RTE_RESULT relation, rather than being totally empty.
*/
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (rte->rtekind == RTE_RELATION)
- return false;
- }
+ if (!bms_is_empty(plannedstmt->lockrels))
+ return false;
}
/*
@@ -1737,17 +1800,22 @@ QueryListGetPrimaryStmt(List *stmts)
/*
* AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
+ *
+ * Returns a list of ExecLockRelsInfo nodes containing one element for each
+ * PlannedStmt in stmt_list; NULL when the latter is utility statement or
+ * its containsInitialPruning is false.
*/
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
+static List *
+AcquireExecutorLocks(List *stmt_list, ParamListInfo boundParams)
{
ListCell *lc1;
+ List *execlockrelsinfo_list = NIL;
foreach(lc1, stmt_list)
{
PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
+ ExecLockRelsInfo *execlockrelsinfo = NULL;
+ int rti;
if (plannedstmt->commandType == CMD_UTILITY)
{
@@ -1761,27 +1829,113 @@ AcquireExecutorLocks(List *stmt_list, bool acquire)
Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
if (query)
- ScanQueryForLocks(query, acquire);
- continue;
+ ScanQueryForLocks(query, true);
}
-
- foreach(lc2, plannedstmt->rtable)
+ else
{
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (rte->rtekind != RTE_RELATION)
- continue;
+ Bitmapset *lockrels;
/*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
+ * Figure out the set of relations that would need to be locked
+ * before executing the plan.
*/
- if (acquire)
+ if (!plannedstmt->containsInitialPruning)
+ {
+ /*
+ * If the plan contains no initial pruning steps, the executor
+ * would just need to lock whatever relations the planner would
+ * have locked to make the plan.
+ */
+ lockrels = plannedstmt->lockrels;
+ }
+ else
+ {
+ /*
+ * Ask the executor to perform initial pruning steps to skip
+ * relations that are pruned away.
+ */
+ execlockrelsinfo = ExecutorGetLockRels(plannedstmt, boundParams);
+ lockrels = execlockrelsinfo->lockrels;
+ }
+
+ rti = -1;
+ while ((rti = bms_next_member(lockrels, rti)) >= 0)
+ {
+ RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable);
+
+ Assert(rte->rtekind == RTE_RELATION);
+
+ /*
+ * Acquire the appropriate type of lock on each relation OID.
+ * Note that we don't actually try to open the rel, and hence
+ * will not fail if it's been dropped entirely --- we'll just
+ * transiently acquire a non-conflicting lock.
+ */
LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+
+ /*
+ * Remember ExecLockRelsInfo for later adding to the QueryDesc that
+ * will be passed to the executor when executing this plan. May be
+ * NULL, but must keep the list the same length as stmt_list.
+ */
+ execlockrelsinfo_list = lappend(execlockrelsinfo_list,
+ execlockrelsinfo);
+ }
+
+ return execlockrelsinfo_list;
+}
+
+/*
+ * ReleaseExecutorLocks
+ * Release locks that would've been acquired by an earlier call to
+ * AcquireExecutorLocks()
+ */
+static void
+ReleaseExecutorLocks(List *stmt_list, List *execlockrelsinfo_list)
+{
+ ListCell *lc1,
+ *lc2;
+
+ forboth(lc1, stmt_list, lc2, execlockrelsinfo_list)
+ {
+ PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
+ ExecLockRelsInfo *execlockrelsinfo = lfirst_node(ExecLockRelsInfo, lc2);
+ int rti;
+
+ if (plannedstmt->commandType == CMD_UTILITY)
+ {
+ /*
+ * Ignore utility statements, except those (such as EXPLAIN) that
+ * contain a parsed-but-not-planned query. Note: it's okay to use
+ * ScanQueryForLocks, even though the query hasn't been through
+ * rule rewriting, because rewriting doesn't change the query
+ * representation.
+ */
+ Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
+
+ if (query)
+ ScanQueryForLocks(query, false);
+ }
+ else
+ {
+ Bitmapset *lockrels;
+
+ if (execlockrelsinfo == NULL)
+ lockrels = plannedstmt->lockrels;
else
+ lockrels = execlockrelsinfo->lockrels;
+
+ rti = -1;
+ while ((rti = bms_next_member(lockrels, rti)) >= 0)
+ {
+ RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable);
+
+ Assert(rte->rtekind == RTE_RELATION);
+
UnlockRelationOid(rte->relid, rte->rellockmode);
+ }
}
}
}
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index d549f66d4a..896f51be08 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -285,6 +285,7 @@ PortalDefineQuery(Portal portal,
const char *sourceText,
CommandTag commandTag,
List *stmts,
+ List *execlockrelsinfos,
CachedPlan *cplan)
{
AssertArg(PortalIsValid(portal));
@@ -299,6 +300,7 @@ PortalDefineQuery(Portal portal,
portal->qc.nprocessed = 0;
portal->commandTag = commandTag;
portal->stmts = stmts;
+ portal->execlockrelsinfos = execlockrelsinfos;
portal->cplan = cplan;
portal->status = PORTAL_DEFINED;
}
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 666977fb1f..fef75ba147 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -87,7 +87,8 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern void ExplainOnePlan(PlannedStmt *plannedstmt, ExecLockRelsInfo *execlockrelsinfo,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h
index fd5735a946..ded19b8cbb 100644
--- a/src/include/executor/execPartition.h
+++ b/src/include/executor/execPartition.h
@@ -124,4 +124,6 @@ extern PartitionPruneState *ExecInitPartitionPruning(PlanState *planstate,
PartitionPruneInfo *pruneinfo,
Bitmapset **initially_valid_subplans);
extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate);
+extern Bitmapset *ExecGetLockRelsDoInitialPruning(Plan *plan, ExecGetLockRelsContext *context,
+ PartitionPruneInfo *pruneinfo);
#endif /* EXECPARTITION_H */
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index e79e2c001f..4338463479 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -35,6 +35,7 @@ typedef struct QueryDesc
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ ExecLockRelsInfo *execlockrelsinfo; /* ExecutorGetLockRels()'s output given plannedstmt */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +58,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ ExecLockRelsInfo *execlockrelsinfo,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 344399f6a8..5959d67221 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -185,6 +185,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
+extern ExecLockRelsInfo *ExecutorGetLockRels(PlannedStmt *plannedstmt, ParamListInfo params);
+extern bool ExecGetLockRels(Plan *node, ExecGetLockRelsContext *context);
extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/executor/nodeAppend.h b/src/include/executor/nodeAppend.h
index 4cb78ee5b6..b53535c2a4 100644
--- a/src/include/executor/nodeAppend.h
+++ b/src/include/executor/nodeAppend.h
@@ -17,6 +17,7 @@
#include "access/parallel.h"
#include "nodes/execnodes.h"
+extern bool ExecGetAppendLockRels(Append *node, ExecGetLockRelsContext *context);
extern AppendState *ExecInitAppend(Append *node, EState *estate, int eflags);
extern void ExecEndAppend(AppendState *node);
extern void ExecReScanAppend(AppendState *node);
diff --git a/src/include/executor/nodeMergeAppend.h b/src/include/executor/nodeMergeAppend.h
index 97fe3b0665..8eb4e9df93 100644
--- a/src/include/executor/nodeMergeAppend.h
+++ b/src/include/executor/nodeMergeAppend.h
@@ -16,6 +16,7 @@
#include "nodes/execnodes.h"
+extern bool ExecGetMergeAppendLockRels(MergeAppend *node, ExecGetLockRelsContext *context);
extern MergeAppendState *ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags);
extern void ExecEndMergeAppend(MergeAppendState *node);
extern void ExecReScanMergeAppend(MergeAppendState *node);
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 1d225bc88d..5006499088 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -19,6 +19,7 @@ extern void ExecComputeStoredGenerated(ResultRelInfo *resultRelInfo,
EState *estate, TupleTableSlot *slot,
CmdType cmdtype);
+extern bool ExecGetModifyTableLockRels(ModifyTable *plan, ExecGetLockRelsContext *context);
extern ModifyTableState *ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags);
extern void ExecEndModifyTable(ModifyTableState *node);
extern void ExecReScanModifyTable(ModifyTableState *node);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index dd95dc40c7..718603d400 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -570,6 +570,7 @@ typedef struct EState
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct ExecLockRelsInfo *es_execlockrelsinfo; /* QueryDesc.execlockrelsinfo */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
@@ -958,6 +959,92 @@ typedef struct DomainConstraintState
*/
typedef TupleTableSlot *(*ExecProcNodeMtd) (struct PlanState *pstate);
+/*----------------
+ * ExecLockRelsInfo
+ *
+ * Result of performing ExecutorGetLockRels() for a given PlannedStmt
+ */
+typedef struct ExecLockRelsInfo
+{
+ NodeTag type;
+
+ /*
+ * Relations that must be locked to execute the plan tree contained in
+ * the PlannedStmt.
+ */
+ Bitmapset *lockrels;
+
+ /* PlannedStmt.numPlanNodes */
+ int numPlanNodes;
+
+ /*
+ * List of PlanInitPruningOutput, each representing the output of
+ * performing initial pruning on a given plan node, for all nodes in the
+ * plan tree that have been marked as needing initial pruning.
+ *
+ * 'ipoIndexes' is an array of 'numPlanNodes' elements, indexed with
+ * plan_node_id of the individual nodes in the plan tree, each a 1-based
+ * index into 'initPruningOutputs' list for a given plan node. 0 means
+ * that a given plan node has no entry in the list because of not needing
+ * any initial pruning done on it.
+ */
+ List *initPruningOutputs;
+ int *ipoIndexes;
+} ExecLockRelsInfo;
+
+/*----------------
+ * ExecGetLockRelsContext
+ *
+ * Context information for performing ExecutorGetLockRels() on a given plan
+ */
+typedef struct ExecGetLockRelsContext
+{
+ NodeTag type;
+
+ PlannedStmt *stmt; /* target plan */
+ ParamListInfo params; /* EXTERN parameters to prune with */
+
+ /* Output parameters for ExecGetLockRels and its subroutines. */
+ Bitmapset *lockrels;
+
+ /* See above comment. */
+ List *initPruningOutputs;
+ int *ipoIndexes;
+} ExecGetLockRelsContext;
+
+#define ExecStorePlanInitPruningOutput(prepcxt, initPruningOutput, plannode) \
+ do { \
+ (prepcxt)->initPruningOutputs = lappend((prepcxt)->initPruningOutputs, initPruningOutput); \
+ (prepcxt)->ipoIndexes[(plannode)->plan_node_id] = list_length((prepcxt)->initPruningOutputs); \
+ } while (0)
+
+#define ExecFetchPlanInitPruningOutput(prepres, plannode) \
+ (((prepres) != NULL && (prepres)->initPruningOutputs != NIL) ? \
+ list_nth((prepres)->initPruningOutputs, \
+ (prepres)->ipoIndexes[(plannode)->plan_node_id] - 1) : NULL)
+
+/* ---------------
+ * PlanInitPruningOutput
+ *
+ * Node to remember the result of performing initial partition pruning steps
+ * during ExecutorGetLockRels() on nodes that support pruning.
+ *
+ * ExecLockRelsDoInitPruning(), which runs during ExecutorGetLockRels(),
+ * creates it and stores it in the corresponding ExecLockRelsInfo.
+ *
+ * ExecInitPartitionPruning(), which runs during ExecuorStart(), fetches it
+ * from the EState's ExecLockRelsInfo (if any) and uses the value of
+ * initially_valid_subplans contained in it as-is to select the subplans to be
+ * initialized for execution, instead of re-evaluating that by performing
+ * initial pruning again.
+ */
+typedef struct PlanInitPruningOutput
+{
+ NodeTag type;
+
+ Bitmapset *initially_valid_subplans;
+} PlanInitPruningOutput;
+
/* ----------------
* PlanState node
*
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5d075f0c34..d365fc4402 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -96,6 +96,11 @@ typedef enum NodeTag
T_PartitionPruneStepCombine,
T_PlanInvalItem,
+ /* TAGS FOR EXECUTOR PREP NODES (execnodes.h) */
+ T_ExecGetLockRelsContext,
+ T_ExecLockRelsInfo,
+ T_PlanInitPruningOutput,
+
/*
* TAGS FOR PLAN STATE NODES (execnodes.h)
*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1f3845b3fe..96c652ebaf 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -101,6 +101,9 @@ typedef struct PlannerGlobal
List *finalrtable; /* "flat" rangetable for executor */
+ Bitmapset *lockrels; /* Indexes of RTE_RELATION entries in range
+ * table */
+
List *finalrowmarks; /* "flat" list of PlanRowMarks */
List *resultRelations; /* "flat" list of integer RT indexes */
@@ -129,6 +132,10 @@ typedef struct PlannerGlobal
char maxParallelHazard; /* worst PROPARALLEL hazard level */
+ bool containsInitialPruning; /* Do some Plan nodes in the tree
+ * have initial (pre-exec) pruning
+ * steps? */
+
PartitionDirectory partition_directory; /* partition descriptors */
} PlannerGlobal;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 0b518ce6b2..5a8c34bdf6 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -59,12 +59,21 @@ typedef struct PlannedStmt
bool parallelModeNeeded; /* parallel mode required to execute? */
+ bool containsInitialPruning; /* Do some Plan nodes in the tree
+ * have initial (pre-exec) pruning
+ * steps? */
+
int jitFlags; /* which forms of JIT should be performed */
struct Plan *planTree; /* tree of Plan nodes */
+ int numPlanNodes; /* number of nodes in planTree */
+
List *rtable; /* list of RangeTblEntry nodes */
+ Bitmapset *lockrels; /* Indexes of RTE_RELATION entries in range
+ * table */
+
/* rtable indexes of target relations for INSERT/UPDATE/DELETE */
List *resultRelations; /* integer list of RT indexes, or NIL */
@@ -1172,6 +1181,13 @@ typedef struct PlanRowMark
* prune_infos List of Lists containing PartitionedRelPruneInfo nodes,
* one sublist per run-time-prunable partition hierarchy
* appearing in the parent plan node's subplans.
+ *
+ * needs_init_pruning Does any of the PartitionedRelPruneInfos in
+ * prune_infos have its initial_pruning_steps set?
+ *
+ * needs_exec_pruning Does any of the PartitionedRelPruneInfos in
+ * prune_infos have its exec_pruning_steps set?
+ *
* other_subplans Indexes of any subplans that are not accounted for
* by any of the PartitionedRelPruneInfo nodes in
* "prune_infos". These subplans must not be pruned.
@@ -1180,6 +1196,8 @@ typedef struct PartitionPruneInfo
{
NodeTag type;
List *prune_infos;
+ bool needs_init_pruning;
+ bool needs_exec_pruning;
Bitmapset *other_subplans;
} PartitionPruneInfo;
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 92291a750d..bf80c53bed 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -64,7 +64,7 @@ extern PlannedStmt *pg_plan_query(Query *querytree, const char *query_string,
ParamListInfo boundParams);
extern List *pg_plan_queries(List *querytrees, const char *query_string,
int cursorOptions,
- ParamListInfo boundParams);
+ ParamListInfo boundParams, List **execlockrelsinfo_list);
extern bool check_max_stack_depth(int *newval, void **extra, GucSource source);
extern void assign_max_stack_depth(int newval, void *extra);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 95b99e3d25..2a847f54da 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -148,6 +148,9 @@ typedef struct CachedPlan
{
int magic; /* should equal CACHEDPLAN_MAGIC */
List *stmt_list; /* list of PlannedStmts */
+ List *execlockrelsinfo_list; /* list of ExecutorGetLockRelsResult with one
+ * element for each of stmt_list; NIL
+ * if not a generic plan */
bool is_oneshot; /* is it a "oneshot" plan? */
bool is_saved; /* is CachedPlan in a long-lived context? */
bool is_valid; /* is the stmt_list currently valid? */
@@ -158,6 +161,8 @@ typedef struct CachedPlan
int generation; /* parent's generation number for this plan */
int refcount; /* count of live references to this struct */
MemoryContext context; /* context containing this CachedPlan */
+ MemoryContext execlockrelsinfo_context; /* context containing execlockrelsinfo_list,
+ * a child of the above context */
} CachedPlan;
/*
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aeddbdafe5..9abace6734 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -137,6 +137,10 @@ typedef struct PortalData
CommandTag commandTag; /* command tag for original query */
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
+ List *execlockrelsinfos; /* list of ExecutorGetLockRelsResults with one element
+ * for each of 'stmts'; same as
+ * cplan->execlockrelsinfo_list if cplan is
+ * not NULL */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
ParamListInfo portalParams; /* params to pass to query */
@@ -241,6 +245,7 @@ extern void PortalDefineQuery(Portal portal,
const char *sourceText,
CommandTag commandTag,
List *stmts,
+ List *execlockrelsinfos,
CachedPlan *cplan);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
--
2.24.1
[application/octet-stream] v5-0001-Some-refactoring-of-runtime-pruning-code.patch (26.4K, 4-v5-0001-Some-refactoring-of-runtime-pruning-code.patch)
download | inline diff:
From 1164015d8561151d1fb5d861b236961e237102ff Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Wed, 2 Mar 2022 15:17:55 +0900
Subject: [PATCH v5 1/3] Some refactoring of runtime pruning code
This does two things mainly:
* Move the execution pruning initialization steps that are common
between both ExecInitAppend() and ExecInitMergeAppend() into a new
function ExecInitPartitionPruning() defined in execPartition.c.
Thus, ExecFindInitialMatchingSubPlans() need not be exported.
* Add an ExprContext field to PartitionPruneContext to remove the
implicit assumption in the runtime pruning code that the ExprContext
to use to compute pruning expressions that need one can always rely
on the PlanState providing it. A future patch will allow runtime
pruning (at least the initial pruning steps) to be performed without
the corresponding PlanState yet having been created, so this will
help.
---
src/backend/executor/execPartition.c | 340 ++++++++++++++++---------
src/backend/executor/nodeAppend.c | 33 +--
src/backend/executor/nodeMergeAppend.c | 32 +--
src/backend/partitioning/partprune.c | 20 +-
src/include/executor/execPartition.h | 9 +-
src/include/partitioning/partprune.h | 2 +
6 files changed, 255 insertions(+), 181 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 90ed1485d1..21953f253b 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -182,11 +182,18 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel,
bool *isnull,
int maxfieldlen);
static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri);
+static PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
+ PartitionPruneInfo *partitionpruneinfo);
+static Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate);
static void ExecInitPruningContext(PartitionPruneContext *context,
List *pruning_steps,
PartitionDesc partdesc,
PartitionKey partkey,
- PlanState *planstate);
+ PlanState *planstate,
+ ExprContext *econtext);
+static void ExecPartitionPruneFixSubPlanIndexes(PartitionPruneState *prunestate,
+ Bitmapset *initially_valid_subplans,
+ int n_total_subplans);
static void find_matching_subplans_recurse(PartitionPruningData *prunedata,
PartitionedRelPruningData *pprune,
bool initial_prune,
@@ -1485,30 +1492,87 @@ adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri)
*
* Functions:
*
- * ExecCreatePartitionPruneState:
- * Creates the PartitionPruneState required by each of the two pruning
- * functions. Details stored include how to map the partition index
- * returned by the partition pruning code into subplan indexes.
- *
- * ExecFindInitialMatchingSubPlans:
- * Returns indexes of matching subplans. Partition pruning is attempted
- * without any evaluation of expressions containing PARAM_EXEC Params.
- * This function must be called during executor startup for the parent
- * plan before the subplans themselves are initialized. Subplans which
- * are found not to match by this function must be removed from the
- * plan's list of subplans during execution, as this function performs a
- * remap of the partition index to subplan index map and the newly
- * created map provides indexes only for subplans which remain after
- * calling this function.
+ * ExecInitPartitionPruning:
+ * Sets up run-time pruning data structure (PartitionPruneState) that is
+ * needed by each of the two pruning functions. Also determines the set
+ * of initially valid subplans by performing initial pruning steps,
+ * telling the caller (such as ExecInitAppend) to initialize only those
+ * for execution. Maps in PartitionPruneState that are used to map the
+ * partition indexes returned by partprune.c functions into the indexes
+ * of partition's subplans in the parent node (such as Append) are
+ * updated to account for initial pruning having eliminated some of the
+ * subplans, if any.
*
* ExecFindMatchingSubPlans:
* Returns indexes of matching subplans after evaluating all available
- * expressions. This function can only be called during execution and
- * must be called again each time the value of a Param listed in
- * PartitionPruneState's 'execparamids' changes.
+ * expressions, that is, using execution pruning steps. This function can
+ * can only be called during execution and must be called again each time
+ * the value of a Param listed in PartitionPruneState's 'execparamids'
+ * changes.
*-------------------------------------------------------------------------
*/
+/*
+ * ExecInitPartitionPruning
+ * Initialize data structure needed for run-time partition pruning
+ *
+ * Initial pruning can be done immediately, so it is done here if needed and
+ * the set of surviving partition subplans' indexes are added to the output
+ * parameter *initially_valid_subplans. If subplans are indeed pruned,
+ * subplan_map arrays contained in the returned PartitionPruneState are
+ * re-sequenced to not count those, though only if the maps will be needed
+ * for subsequent execution pruning passes.
+ */
+PartitionPruneState *
+ExecInitPartitionPruning(PlanState *planstate,
+ int n_total_subplans,
+ PartitionPruneInfo *pruneinfo,
+ Bitmapset **initially_valid_subplans)
+{
+ PartitionPruneState *prunestate;
+ EState *estate = planstate->state;
+
+ /* We may need an expression context to evaluate partition exprs */
+ ExecAssignExprContext(estate, planstate);
+
+ /*
+ * Create the working data structure for pruning.
+ */
+ prunestate = ExecCreatePartitionPruneState(planstate, pruneinfo);
+
+ /*
+ * Perform an initial partition prune, if required.
+ */
+ if (prunestate->do_initial_prune)
+ {
+ /* Determine which subplans survive initial pruning */
+ *initially_valid_subplans = ExecFindInitialMatchingSubPlans(prunestate);
+ }
+ else
+ {
+ /* We'll need to initialize all subplans */
+ Assert(n_total_subplans > 0);
+ *initially_valid_subplans = bms_add_range(NULL, 0,
+ n_total_subplans - 1);
+ }
+
+ /*
+ * Re-sequence subplan indexes contained in prunestate to account for any
+ * that were removed above due to initial pruning.
+ *
+ * We can safely skip this when !do_exec_prune, even though that leaves
+ * invalid data in prunestate, because that data won't be consulted again
+ * (cf initial Assert in ExecFindMatchingSubPlans).
+ */
+ if (prunestate->do_exec_prune &&
+ bms_num_members(*initially_valid_subplans) < n_total_subplans)
+ ExecPartitionPruneFixSubPlanIndexes(prunestate,
+ *initially_valid_subplans,
+ n_total_subplans);
+
+ return prunestate;
+}
+
/*
* ExecCreatePartitionPruneState
* Build the data structure required for calling
@@ -1527,7 +1591,7 @@ adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri)
* re-used each time we re-evaluate which partitions match the pruning steps
* provided in each PartitionedRelPruneInfo.
*/
-PartitionPruneState *
+static PartitionPruneState *
ExecCreatePartitionPruneState(PlanState *planstate,
PartitionPruneInfo *partitionpruneinfo)
{
@@ -1536,6 +1600,7 @@ ExecCreatePartitionPruneState(PlanState *planstate,
int n_part_hierarchies;
ListCell *lc;
int i;
+ ExprContext *econtext = planstate->ps_ExprContext;
/* For data reading, executor always omits detached partitions */
if (estate->es_partition_directory == NULL)
@@ -1709,7 +1774,8 @@ ExecCreatePartitionPruneState(PlanState *planstate,
{
ExecInitPruningContext(&pprune->initial_context,
pinfo->initial_pruning_steps,
- partdesc, partkey, planstate);
+ partdesc, partkey, planstate,
+ econtext);
/* Record whether initial pruning is needed at any level */
prunestate->do_initial_prune = true;
}
@@ -1718,7 +1784,8 @@ ExecCreatePartitionPruneState(PlanState *planstate,
{
ExecInitPruningContext(&pprune->exec_context,
pinfo->exec_pruning_steps,
- partdesc, partkey, planstate);
+ partdesc, partkey, planstate,
+ econtext);
/* Record whether exec pruning is needed at any level */
prunestate->do_exec_prune = true;
}
@@ -1746,7 +1813,8 @@ ExecInitPruningContext(PartitionPruneContext *context,
List *pruning_steps,
PartitionDesc partdesc,
PartitionKey partkey,
- PlanState *planstate)
+ PlanState *planstate,
+ ExprContext *econtext)
{
int n_steps;
int partnatts;
@@ -1767,6 +1835,7 @@ ExecInitPruningContext(PartitionPruneContext *context,
context->ppccontext = CurrentMemoryContext;
context->planstate = planstate;
+ context->exprcontext = econtext;
/* Initialize expression state for each expression we need */
context->exprstates = (ExprState **)
@@ -1795,8 +1864,20 @@ ExecInitPruningContext(PartitionPruneContext *context,
step->step.step_id,
keyno);
- context->exprstates[stateidx] =
- ExecInitExpr(expr, context->planstate);
+ /*
+ * When planstate is NULL, pruning_steps is known not to
+ * contain any expressions that depend on the parent plan.
+ * Information of any available EXTERN parameters must be
+ * passed explicitly in that case, which the caller must
+ * have made available via econtext.
+ */
+ if (planstate == NULL)
+ context->exprstates[stateidx] =
+ ExecInitExprWithParams(expr,
+ econtext->ecxt_param_list_info);
+ else
+ context->exprstates[stateidx] =
+ ExecInitExpr(expr, context->planstate);
}
keyno++;
}
@@ -1816,11 +1897,9 @@ ExecInitPruningContext(PartitionPruneContext *context,
*
* Must only be called once per 'prunestate', and only if initial pruning
* is required.
- *
- * 'nsubplans' must be passed as the total number of unpruned subplans.
*/
-Bitmapset *
-ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate, int nsubplans)
+static Bitmapset *
+ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate)
{
Bitmapset *result = NULL;
MemoryContext oldcontext;
@@ -1845,14 +1924,20 @@ ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate, int nsubplans)
PartitionedRelPruningData *pprune;
prunedata = prunestate->partprunedata[i];
+
+ /*
+ * We pass the 1st item belonging to the root table of the hierarchy
+ * and find_matching_subplans_recurse() takes care of recursing to
+ * other (lower-level) parents as needed.
+ */
pprune = &prunedata->partrelprunedata[0];
/* Perform pruning without using PARAM_EXEC Params */
find_matching_subplans_recurse(prunedata, pprune, true, &result);
- /* Expression eval may have used space in node's ps_ExprContext too */
+ /* Expression eval may have used space in ExprContext too */
if (pprune->initial_pruning_steps)
- ResetExprContext(pprune->initial_context.planstate->ps_ExprContext);
+ ResetExprContext(pprune->initial_context.exprcontext);
}
/* Add in any subplans that partition pruning didn't account for */
@@ -1865,118 +1950,120 @@ ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate, int nsubplans)
MemoryContextReset(prunestate->prune_context);
+ return result;
+}
+
+/*
+ * ExecPartitionPruneFixSubPlanIndexes
+ * Fix mapping of partition indexes to subplan indexes contained in
+ * prunestate by considering the new list of subplans that survived
+ * initial pruning
+ *
+ * Subplans would be previously indexed 0..(n_total_subplans - 1), though
+ * now should be changed to index range 0..num(initially_valid_subplans).
+ */
+static void
+ExecPartitionPruneFixSubPlanIndexes(PartitionPruneState *prunestate,
+ Bitmapset *initially_valid_subplans,
+ int n_total_subplans)
+{
+ int *new_subplan_indexes;
+ Bitmapset *new_other_subplans;
+ int i;
+ int newidx;
+
/*
- * If exec-time pruning is required and we pruned subplans above, then we
- * must re-sequence the subplan indexes so that ExecFindMatchingSubPlans
- * properly returns the indexes from the subplans which will remain after
- * execution of this function.
- *
- * We can safely skip this when !do_exec_prune, even though that leaves
- * invalid data in prunestate, because that data won't be consulted again
- * (cf initial Assert in ExecFindMatchingSubPlans).
+ * First we must build a temporary array which maps old subplan
+ * indexes to new ones. For convenience of initialization, we use
+ * 1-based indexes in this array and leave pruned items as 0.
*/
- if (prunestate->do_exec_prune && bms_num_members(result) < nsubplans)
+ new_subplan_indexes = (int *) palloc0(sizeof(int) * n_total_subplans);
+ newidx = 1;
+ i = -1;
+ while ((i = bms_next_member(initially_valid_subplans, i)) >= 0)
{
- int *new_subplan_indexes;
- Bitmapset *new_other_subplans;
- int i;
- int newidx;
+ Assert(i < n_total_subplans);
+ new_subplan_indexes[i] = newidx++;
+ }
- /*
- * First we must build a temporary array which maps old subplan
- * indexes to new ones. For convenience of initialization, we use
- * 1-based indexes in this array and leave pruned items as 0.
- */
- new_subplan_indexes = (int *) palloc0(sizeof(int) * nsubplans);
- newidx = 1;
- i = -1;
- while ((i = bms_next_member(result, i)) >= 0)
- {
- Assert(i < nsubplans);
- new_subplan_indexes[i] = newidx++;
- }
+ /*
+ * Now we can update each PartitionedRelPruneInfo's subplan_map with
+ * new subplan indexes. We must also recompute its present_parts
+ * bitmap.
+ */
+ for (i = 0; i < prunestate->num_partprunedata; i++)
+ {
+ PartitionPruningData *prunedata = prunestate->partprunedata[i];
+ int j;
/*
- * Now we can update each PartitionedRelPruneInfo's subplan_map with
- * new subplan indexes. We must also recompute its present_parts
- * bitmap.
+ * Within each hierarchy, we perform this loop in back-to-front
+ * order so that we determine present_parts for the lowest-level
+ * partitioned tables first. This way we can tell whether a
+ * sub-partitioned table's partitions were entirely pruned so we
+ * can exclude it from the current level's present_parts.
*/
- for (i = 0; i < prunestate->num_partprunedata; i++)
+ for (j = prunedata->num_partrelprunedata - 1; j >= 0; j--)
{
- PartitionPruningData *prunedata = prunestate->partprunedata[i];
- int j;
+ PartitionedRelPruningData *pprune = &prunedata->partrelprunedata[j];
+ int nparts = pprune->nparts;
+ int k;
- /*
- * Within each hierarchy, we perform this loop in back-to-front
- * order so that we determine present_parts for the lowest-level
- * partitioned tables first. This way we can tell whether a
- * sub-partitioned table's partitions were entirely pruned so we
- * can exclude it from the current level's present_parts.
- */
- for (j = prunedata->num_partrelprunedata - 1; j >= 0; j--)
- {
- PartitionedRelPruningData *pprune = &prunedata->partrelprunedata[j];
- int nparts = pprune->nparts;
- int k;
+ /* We just rebuild present_parts from scratch */
+ bms_free(pprune->present_parts);
+ pprune->present_parts = NULL;
- /* We just rebuild present_parts from scratch */
- bms_free(pprune->present_parts);
- pprune->present_parts = NULL;
+ for (k = 0; k < nparts; k++)
+ {
+ int oldidx = pprune->subplan_map[k];
+ int subidx;
- for (k = 0; k < nparts; k++)
+ /*
+ * If this partition existed as a subplan then change the
+ * old subplan index to the new subplan index. The new
+ * index may become -1 if the partition was pruned above,
+ * or it may just come earlier in the subplan list due to
+ * some subplans being removed earlier in the list. If
+ * it's a subpartition, add it to present_parts unless
+ * it's entirely pruned.
+ */
+ if (oldidx >= 0)
{
- int oldidx = pprune->subplan_map[k];
- int subidx;
-
- /*
- * If this partition existed as a subplan then change the
- * old subplan index to the new subplan index. The new
- * index may become -1 if the partition was pruned above,
- * or it may just come earlier in the subplan list due to
- * some subplans being removed earlier in the list. If
- * it's a subpartition, add it to present_parts unless
- * it's entirely pruned.
- */
- if (oldidx >= 0)
- {
- Assert(oldidx < nsubplans);
- pprune->subplan_map[k] = new_subplan_indexes[oldidx] - 1;
+ Assert(oldidx < n_total_subplans);
+ pprune->subplan_map[k] = new_subplan_indexes[oldidx] - 1;
- if (new_subplan_indexes[oldidx] > 0)
- pprune->present_parts =
- bms_add_member(pprune->present_parts, k);
- }
- else if ((subidx = pprune->subpart_map[k]) >= 0)
- {
- PartitionedRelPruningData *subprune;
+ if (new_subplan_indexes[oldidx] > 0)
+ pprune->present_parts =
+ bms_add_member(pprune->present_parts, k);
+ }
+ else if ((subidx = pprune->subpart_map[k]) >= 0)
+ {
+ PartitionedRelPruningData *subprune;
- subprune = &prunedata->partrelprunedata[subidx];
+ subprune = &prunedata->partrelprunedata[subidx];
- if (!bms_is_empty(subprune->present_parts))
- pprune->present_parts =
- bms_add_member(pprune->present_parts, k);
- }
+ if (!bms_is_empty(subprune->present_parts))
+ pprune->present_parts =
+ bms_add_member(pprune->present_parts, k);
}
}
}
+ }
- /*
- * We must also recompute the other_subplans set, since indexes in it
- * may change.
- */
- new_other_subplans = NULL;
- i = -1;
- while ((i = bms_next_member(prunestate->other_subplans, i)) >= 0)
- new_other_subplans = bms_add_member(new_other_subplans,
- new_subplan_indexes[i] - 1);
-
- bms_free(prunestate->other_subplans);
- prunestate->other_subplans = new_other_subplans;
+ /*
+ * We must also recompute the other_subplans set, since indexes in it
+ * may change.
+ */
+ new_other_subplans = NULL;
+ i = -1;
+ while ((i = bms_next_member(prunestate->other_subplans, i)) >= 0)
+ new_other_subplans = bms_add_member(new_other_subplans,
+ new_subplan_indexes[i] - 1);
- pfree(new_subplan_indexes);
- }
+ bms_free(prunestate->other_subplans);
+ prunestate->other_subplans = new_other_subplans;
- return result;
+ pfree(new_subplan_indexes);
}
/*
@@ -2018,11 +2105,16 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate)
prunedata = prunestate->partprunedata[i];
pprune = &prunedata->partrelprunedata[0];
+ /*
+ * We pass the 1st item belonging to the root table of the hierarchy
+ * and find_matching_subplans_recurse() takes care of recursing to
+ * other (lower-level) parents as needed.
+ */
find_matching_subplans_recurse(prunedata, pprune, false, &result);
- /* Expression eval may have used space in node's ps_ExprContext too */
+ /* Expression eval may have used space in ExprContext too */
if (pprune->exec_pruning_steps)
- ResetExprContext(pprune->exec_context.planstate->ps_ExprContext);
+ ResetExprContext(pprune->exec_context.exprcontext);
}
/* Add in any subplans that partition pruning didn't account for */
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 7937f1c88f..5b6d3eb23b 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -138,30 +138,17 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
{
PartitionPruneState *prunestate;
- /* We may need an expression context to evaluate partition exprs */
- ExecAssignExprContext(estate, &appendstate->ps);
-
- /* Create the working data structure for pruning. */
- prunestate = ExecCreatePartitionPruneState(&appendstate->ps,
- node->part_prune_info);
+ /*
+ * Set up pruning data structure. Initial pruning steps, if any, are
+ * performed as part of the setup, adding the set of indexes of
+ * surviving subplans to 'validsubplans'.
+ */
+ prunestate = ExecInitPartitionPruning(&appendstate->ps,
+ list_length(node->appendplans),
+ node->part_prune_info,
+ &validsubplans);
appendstate->as_prune_state = prunestate;
-
- /* Perform an initial partition prune, if required. */
- if (prunestate->do_initial_prune)
- {
- /* Determine which subplans survive initial pruning */
- validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
- list_length(node->appendplans));
-
- nplans = bms_num_members(validsubplans);
- }
- else
- {
- /* We'll need to initialize all subplans */
- nplans = list_length(node->appendplans);
- Assert(nplans > 0);
- validsubplans = bms_add_range(NULL, 0, nplans - 1);
- }
+ nplans = bms_num_members(validsubplans);
/*
* When no run-time pruning is required and there's at least one
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 418f89dea8..9a9f29e845 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -86,29 +86,17 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
{
PartitionPruneState *prunestate;
- /* We may need an expression context to evaluate partition exprs */
- ExecAssignExprContext(estate, &mergestate->ps);
-
- prunestate = ExecCreatePartitionPruneState(&mergestate->ps,
- node->part_prune_info);
+ /*
+ * Set up pruning data structure. Initial pruning steps, if any, are
+ * performed as part of the setup, adding the set of indexes of
+ * surviving subplans to 'validsubplans'.
+ */
+ prunestate = ExecInitPartitionPruning(&mergestate->ps,
+ list_length(node->mergeplans),
+ node->part_prune_info,
+ &validsubplans);
mergestate->ms_prune_state = prunestate;
-
- /* Perform an initial partition prune, if required. */
- if (prunestate->do_initial_prune)
- {
- /* Determine which subplans survive initial pruning */
- validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
- list_length(node->mergeplans));
-
- nplans = bms_num_members(validsubplans);
- }
- else
- {
- /* We'll need to initialize all subplans */
- nplans = list_length(node->mergeplans);
- Assert(nplans > 0);
- validsubplans = bms_add_range(NULL, 0, nplans - 1);
- }
+ nplans = bms_num_members(validsubplans);
/*
* When no run-time pruning is required and there's at least one
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 1bc00826c1..7080cb25d9 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -798,6 +798,7 @@ prune_append_rel_partitions(RelOptInfo *rel)
/* These are not valid when being called from the planner */
context.planstate = NULL;
+ context.exprcontext = NULL;
context.exprstates = NULL;
/* Actual pruning happens here. */
@@ -808,8 +809,8 @@ prune_append_rel_partitions(RelOptInfo *rel)
* get_matching_partitions
* Determine partitions that survive partition pruning
*
- * Note: context->planstate must be set to a valid PlanState when the
- * pruning_steps were generated with a target other than PARTTARGET_PLANNER.
+ * Note: context->exprcontext must be valid when the pruning_steps were
+ * generated with a target other than PARTTARGET_PLANNER.
*
* Returns a Bitmapset of the RelOptInfo->part_rels indexes of the surviving
* partitions.
@@ -3654,7 +3655,7 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
* exprstate array.
*
* Note that the evaluated result may be in the per-tuple memory context of
- * context->planstate->ps_ExprContext, and we may have leaked other memory
+ * context->exprcontext, and we may have leaked other memory
* there too. This memory must be recovered by resetting that ExprContext
* after we're done with the pruning operation (see execPartition.c).
*/
@@ -3677,13 +3678,18 @@ partkey_datum_from_expr(PartitionPruneContext *context,
ExprContext *ectx;
/*
- * We should never see a non-Const in a step unless we're running in
- * the executor.
+ * We should never see a non-Const in a step unless the caller has
+ * passed a valid ExprContext.
+ *
+ * When context->planstate is valid, context->exprcontext is same
+ * as context->planstate->ps_ExprContext.
*/
- Assert(context->planstate != NULL);
+ Assert(context->planstate != NULL || context->exprcontext != NULL);
+ Assert(context->planstate == NULL ||
+ (context->exprcontext == context->planstate->ps_ExprContext));
exprstate = context->exprstates[stateidx];
- ectx = context->planstate->ps_ExprContext;
+ ectx = context->exprcontext;
*value = ExecEvalExprSwitchContext(exprstate, ectx, isnull);
}
}
diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h
index 603d8becc4..fd5735a946 100644
--- a/src/include/executor/execPartition.h
+++ b/src/include/executor/execPartition.h
@@ -119,10 +119,9 @@ extern ResultRelInfo *ExecFindPartition(ModifyTableState *mtstate,
EState *estate);
extern void ExecCleanupTupleRouting(ModifyTableState *mtstate,
PartitionTupleRouting *proute);
-extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
- PartitionPruneInfo *partitionpruneinfo);
+extern PartitionPruneState *ExecInitPartitionPruning(PlanState *planstate,
+ int n_total_subplans,
+ PartitionPruneInfo *pruneinfo,
+ Bitmapset **initially_valid_subplans);
extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate);
-extern Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate,
- int nsubplans);
-
#endif /* EXECPARTITION_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index ee11b6feae..90684efa25 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -41,6 +41,7 @@ struct RelOptInfo;
* subsidiary data, such as the FmgrInfos.
* planstate Points to the parent plan node's PlanState when called
* during execution; NULL when called from the planner.
+ * exprcontext ExprContext to use when evaluating pruning expressions
* exprstates Array of ExprStates, indexed as per PruneCxtStateIdx; one
* for each partition key in each pruning step. Allocated if
* planstate is non-NULL, otherwise NULL.
@@ -56,6 +57,7 @@ typedef struct PartitionPruneContext
FmgrInfo *stepcmpfuncs;
MemoryContext ppccontext;
PlanState *planstate;
+ ExprContext *exprcontext;
ExprState **exprstates;
} PartitionPruneContext;
--
2.24.1
view thread (71+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: generic plans and "initial" pruning
In-Reply-To: <CA+HiwqEYCpEqh2LMDOp9mT+4-QoVe8HgFMKBjntEMCTZLpcCCA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox