public inbox for [email protected]
help / color / mirror / Atom feedRe: generic plans and "initial" pruning
31+ messages / 8 participants
[nested] [flat]
* Re: generic plans and "initial" pruning
@ 2023-04-05 23:23 Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-04-05 23:23 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Tue, Apr 4, 2023 at 10:29 PM Amit Langote <[email protected]>
wrote:
> On Tue, Apr 4, 2023 at 6:41 AM Tom Lane <[email protected]> wrote:
> > A few concrete thoughts:
> >
> > * I understand that your plan now is to acquire locks on all the
> > originally-named tables, then do permissions checks (which will
> > involve only those tables), then dynamically lock just inheritance and
> > partitioning child tables as we descend the plan tree.
>
> Actually, with the current implementation of the patch, *all* of the
> relations mentioned in the plan tree would get locked during the
> ExecInitNode() traversal of the plan tree (and of those in
> plannedstmt->subplans), not just the inheritance child tables.
> Locking of non-child tables done by the executor after this patch is
> duplicative with AcquirePlannerLocks(), so that's something to be
> improved.
>
> > That seems
> > more or less okay to me, but it could be reflected better in the
> > structure of the patch perhaps.
> >
> > * In particular I don't much like the "viewRelations" list, which
> > seems like a wart; those ought to be handled more nearly the same way
> > as other RTEs. (One concrete reason why is that this scheme is going
> > to result in locking views in a different order than they were locked
> > during original parsing, which perhaps could contribute to deadlocks.)
> > Maybe we should store an integer list of which RTIs need to be locked
> > in the early phase? Building that in the parser/rewriter would provide
> > a solid guide to the original locking order, so we'd be trivially sure
> > of duplicating that. (It might be close enough to follow the RT list
> > order, which is basically what AcquireExecutorLocks does today, but
> > this'd be more certain to do the right thing.) I'm less concerned
> > about lock order for child tables because those are just going to
> > follow the inheritance or partitioning structure.
>
> What you've described here sounds somewhat like what I had implemented
> in the patch versions till v31, though it used a bitmapset named
> minLockRelids that is initialized by setrefs.c. Your idea of
> initializing a list before planning seems more appealing offhand than
> the code I had added in setrefs.c to populate that minLockRelids
> bitmapset, which would be bms_add_range(1, list_lenth(finalrtable)),
> followed by bms_del_members(set-of-child-rel-rtis).
>
> I'll give your idea a try.
After sleeping on this, I think we perhaps don't need to remember
originally-named relations if only for the purpose of locking them for
execution. That's because, for a reused (cached) plan,
AcquirePlannerLocks() would have taken those locks anyway.
AcquirePlannerLocks() doesn't lock inheritance children because they would
be added to the range table by the planner, so they should be locked
separately for execution, if needed. I thought taking the execution-time
locks only when inside ExecInit[Merge]Append would work, but then we have
cases where single-child Append/MergeAppend are stripped of the
Append/MergeAppend nodes by setrefs.c. Maybe we need a place to remember
such child relations, that is, only in the cases where Append/MergeAppend
elision occurs, in something maybe esoteric-sounding like
PlannedStmt.elidedAppendChildRels or something?
Another set of child relations that are not covered by Append/MergeAppend
child nodes is non-leaf partitions. I've proposed adding a List of
Bitmapset field to Append/MergeAppend named 'allpartrelids' as part of this
patchset (patch 0001) to track those for execution-time locking.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-06-08 14:23 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-06-08 14:23 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
Here is a new version. Summary of main changes since the last version
that Tom reviewed back in April:
* ExecInitNode() subroutines now return NULL (as opposed to a
partially initialized PlanState node as in the last version) upon
detecting that the CachedPlan that the plan tree is from is no longer
valid due to invalidation messages processed upon taking locks. Plan
tree subnodes that are fully initialized till the point of detection
are added by ExecInitNode() into a List in EState called
es_inited_plannodes. ExecEndPlan() now iterates over that list to
close each one individually using ExecEndNode(). ExecEndNode() or its
subroutines thus no longer need to be recursive to close the child
nodes. Also, with this design, there is no longer the possibility of
partially initialized PlanState trees with partially initialized
individual PlanState nodes, so the ExecEndNode() subroutine changes
that were in the last version to account for partial initialization
are not necessary.
* Instead of setting EXEC_FLAG_GET_LOCKS in es_top_eflags for the
entire duration of InitPlan(), it is now only set in ExecInitAppend()
and ExecInitMergeAppend(), because that's where the subnodes scanning
child tables would be and the executor only needs to lock child tables
to validate a CachedPlan in a race-free manner. Parent tables that
appear in the query would have been locked by AcquirePlannerLocks().
Child tables whose scan subnodes don't appear under Append/MergeAppend
(due to the latter being removed by setrefs.c for there being only a
single child) are identified in PlannedStmt.elidedAppendChildRelations
and InitPlan() locks each one found there if the plan tree is from a
CachedPlan.
* There's no longer PlannedStmt.viewRelations, because view relations
need not be tracked separately for locking as AcquirePlannerLocks()
covers them.
Attachments:
[application/octet-stream] v39-0004-Track-opened-range-table-relations-in-a-List-in-.patch (2.3K, 2-v39-0004-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From b27b16024d8e673062520b8a3792b71d51e1aed9 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Mon, 13 Mar 2023 15:59:38 +0900
Subject: [PATCH v39 4/4] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing 1000s of partition subplans.
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 00db6eb307..28b72213c4 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1649,12 +1649,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 599db4d597..be100f4bd8 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -831,6 +831,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f0c5177b06..be06c40766 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v39-0002-Add-a-PlannedStmt-field-to-store-RT-indexes-of-o.patch (6.4K, 3-v39-0002-Add-a-PlannedStmt-field-to-store-RT-indexes-of-o.patch)
download | inline diff:
From aa3de3f0770cf7f2b91d70de90922fdfce947cf5 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Wed, 7 Jun 2023 21:00:58 +0900
Subject: [PATCH v39 2/4] Add a PlannedStmt field to store RT indexes of
once-child relations
A future commit will teach the executor to lock only the child tables
appearing in a plan and to identify those tables it will rely on the
fact that plan tree subnodes to scan the child tables normally appear
under an Append/MergeAppend node. But when there's only one child
subnode, setrefs.c removes the redundant Append/MergeAppend node,
making it impossible for the executor to identify the subnode as
scanning a child table.
This commit makes setrefs.c store the RT indexes of child tables
scanned by such once-child subnodes into a new field of PlannedStmt
called elidedAppendChildRelations.
There are no users of that field as of this commit but the
aforementioned future commit will use it to lock child tables that
don't appears under Append/MergeAppend.
---
src/backend/optimizer/plan/planner.c | 1 +
src/backend/optimizer/plan/setrefs.c | 97 ++++++++++++++++++++++++++++
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 2 +
4 files changed, 103 insertions(+)
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8e3d2c1e35..27a4c7585a 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -526,6 +526,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->permInfos = glob->finalrteperminfos;
result->resultRelations = glob->resultRelations;
result->appendRelations = glob->appendRelations;
+ result->elidedAppendChildRels = glob->elidedAppendChildRels;
result->subplans = glob->subplans;
result->rewindPlanIDs = glob->rewindPlanIDs;
result->rowMarks = glob->finalrowmarks;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index c4db6812ec..8ccc869bdd 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/transam.h"
+#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
@@ -134,6 +135,7 @@ static void flatten_unplanned_rtes(PlannerGlobal *glob, RangeTblEntry *rte);
static bool flatten_rtes_walker(Node *node, flatten_rtes_walker_context *cxt);
static void add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos,
RangeTblEntry *rte);
+static List *add_plan_scanrelids(List *scanrelids, Plan *plan);
static Plan *set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset);
static Plan *set_indexonlyscan_references(PlannerInfo *root,
IndexOnlyScan *plan,
@@ -601,6 +603,93 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos,
}
}
+/*
+ * Recursively adds the RT index(es) of the relation(s) scanned by plan
+ * to 'scanrelids'.
+ */
+static List *
+add_plan_scanrelids(List *scanrelids, Plan *plan)
+{
+ if (plan == NULL)
+ return scanrelids;
+
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ case T_SampleScan:
+ case T_IndexScan:
+ case T_IndexOnlyScan:
+ case T_BitmapHeapScan:
+ case T_TidScan:
+ case T_TidRangeScan:
+ case T_SubqueryScan:
+ case T_FunctionScan:
+ case T_TableFuncScan:
+ case T_ValuesScan:
+ case T_CteScan:
+ case T_WorkTableScan:
+ case T_NamedTuplestoreScan:
+ case T_ForeignScan:
+ case T_CustomScan:
+ scanrelids = lappend_int(scanrelids, ((Scan *) plan)->scanrelid);
+ break;
+
+ /* Recurse for nodes that have child plans. */
+ case T_Append:
+ {
+ Append *aplan = (Append *) plan;
+ ListCell *l;
+
+ foreach(l, aplan->appendplans)
+ scanrelids = add_plan_scanrelids(scanrelids,
+ (Plan *) lfirst(l));
+ }
+ break;
+
+ case T_MergeAppend:
+ {
+ MergeAppend *mplan = (MergeAppend *) plan;
+ ListCell *l;
+
+ foreach(l, mplan->mergeplans)
+ scanrelids = add_plan_scanrelids(scanrelids,
+ (Plan *) lfirst(l));
+ }
+ break;
+
+ case T_BitmapAnd:
+ {
+ BitmapAnd *baplan = (BitmapAnd *) plan;
+ ListCell *l;
+
+ foreach(l, baplan->bitmapplans)
+ scanrelids = add_plan_scanrelids(scanrelids,
+ (Plan *) lfirst(l));
+ }
+ break;
+
+ case T_BitmapOr:
+ {
+ BitmapOr *boplan = (BitmapOr *) plan;
+ ListCell *l;
+
+ foreach(l, boplan->bitmapplans)
+ scanrelids = add_plan_scanrelids(scanrelids,
+ (Plan *) lfirst(l));
+ }
+ break;
+
+ default:
+ break;
+ }
+
+ /* Recurse into child plans. */
+ scanrelids = add_plan_scanrelids(scanrelids, plan->lefttree);
+ scanrelids = add_plan_scanrelids(scanrelids, plan->righttree);
+
+ return scanrelids;
+}
+
/*
* set_plan_refs: recurse through the Plan nodes of a single subquery level
*/
@@ -1743,7 +1832,11 @@ set_append_references(PlannerInfo *root,
Plan *p = (Plan *) linitial(aplan->appendplans);
if (p->parallel_aware == aplan->plan.parallel_aware)
+ {
+ root->glob->elidedAppendChildRels =
+ add_plan_scanrelids(root->glob->elidedAppendChildRels, p);
return clean_up_removed_plan_level((Plan *) aplan, p);
+ }
}
/*
@@ -1821,7 +1914,11 @@ set_mergeappend_references(PlannerInfo *root,
Plan *p = (Plan *) linitial(mplan->mergeplans);
if (p->parallel_aware == mplan->plan.parallel_aware)
+ {
+ root->glob->elidedAppendChildRels =
+ add_plan_scanrelids(root->glob->elidedAppendChildRels, p);
return clean_up_removed_plan_level((Plan *) mplan, p);
+ }
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..4303482499 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -125,6 +125,9 @@ typedef struct PlannerGlobal
/* "flat" list of AppendRelInfos */
List *appendRelations;
+ /* "flat" list of integer RT indexes */
+ List *elidedAppendChildRels;
+
/* OIDs of relations the plan depends on */
List *relationOids;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7a5f3ba625..203268d1ba 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -80,6 +80,8 @@ typedef struct PlannedStmt
List *appendRelations; /* list of AppendRelInfo nodes */
+ List *elidedAppendChildRels; /* "flat" list of integer RT indexes */
+
List *subplans; /* Plan trees for SubPlan expressions; note
* that some could be NULL */
--
2.35.3
[application/octet-stream] v39-0001-Add-field-to-store-partitioned-relids-to-Append-.patch (21.4K, 4-v39-0001-Add-field-to-store-partitioned-relids-to-Append-.patch)
download | inline diff:
From 6d035cbd208f4b7de978bf4c7fbb7e1f8db07d24 Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Thu, 9 Mar 2023 11:26:06 +0900
Subject: [PATCH v39 1/4] Add field to store partitioned relids to
Append/MergeAppend
A future commit would like to move the timing of locking relations
referenced in a cached plan to ExecInitNode() traversal of the plan
tree from the current loop-over-rangetable in AcquireExecutorLocks().
But partitioned tables do not have their own Scan nodes for the
executor to be able to find them through the plan tree traversal, so
their RT indexes must be remembered via this new field of
Append/MergeAppend node.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but was local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids(). Considering that the partitioned
parent relids must also be looked up for the cases where
Append/MergeAppend child subpaths are not simple scan nodes, but also
join or aggregrate nodes (due to partitionwise join and aggregate
features), the code in new function needs to be generalized to the
cases where child rels can be joinrels or upper (grouping) rels.
Finally, to facilitate the lookup of parent rels in
add_append_subpath_partrelids(), set the link to parent rels in the
RelOptInfos of child grouping rels too, like it's already done for
the RelOptInfos of child baserels (scan-level) and child joinrels.
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 4bb38160b3..48febf4045 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1e4dd27dba..8e3d2c1e35 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7798,8 +7798,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1ca26baa25..c4db6812ec 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1754,6 +1754,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1830,6 +1832,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..14c74a3a4e 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply get the parent relid from
+ * prel->parent. But for partitionwise join and aggregate child rels,
+ * while we can use prel->parent to move up the tree, parent relids to
+ * add into 'partrelids' must be found the hard way through the
+ * AppendInfoInfos, because 1) a joinrel's relids may point to RTE_JOIN
+ * entries, 2) topmost parent grouping rel's relids field is left NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v39-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch (125.5K, 5-v39-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 5a91c233c6bd0c66db36b41a4c3005a6f3a910bd Mon Sep 17 00:00:00 2001
From: amitlan <[email protected]>
Date: Fri, 20 Jan 2023 16:52:31 +0900
Subject: [PATCH v39 3/4] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes locks on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters, many
of them would be locked unnecessarily, because only those that
survive the pruning need to have been locked. Locking all partitions
this way causes significant delay when there are many partitions.
This commit rearranges things so that the child tables are now locked
during the ExecInitNode() initialization of the plan tree in a
CachedPlan. If the locking of child tables causes the CachedPlan
to go stale (that is, its is_valid set to false by
PlanCacheRelCallback() when an invalidation message matching some
child table contained in the plan is processed), the executor abandons
further execution with it and asks the caller to retry with a new one.
When a CachedPlan is found to have gone stale as described above,
QueryDesc.planstate is set to NULL, indicating that no execution is
possible with the plan tree as is. Though some plan tree subnodes
may get fully initialized before the CachedPlan's staleness is
detected, so to ensure that they are released by ExecEndPlan(),
ExecInitNode() now adds successfully initialized nodes a new List
in EState called es_inited_plannodes. ExecEndPlan() releases them
individually by calling ExecEndNode() on each. ExecEndNode() is
no longer recursive, because all nodes that need to be closed are
found by interating es_inited_plannodes.
This commit introduces a new executor flag EXEC_FLAG_GET_LOCKS that
should be added into eflags to indicate that
ExecGetRangeTableRelation() take a lock on the table. It is only
set when in ExecInit[Merge]Append() given that child tables or plan
subnodes to scan them normally appear under Append/MergeAppend.
Those that don't are locked directly in InitPlan() using their RT
indexes found in PlannedStmt.elidedAppendChildRelations.
Call sites that use plancache (GetCachedPlan) to get the plan trees
to pass to the executor for execution should now be prepared to
handle the case that the plan tree may be flagged by the executor as
stale as described above. To that end, this commit refactors the
relevant code sites to move the ExecutorStart() call closer to the
GetCachedPlan() to implement the replan loop conveniently.
PortalStart() now performs CreateQueryDesc() and ExecutorStart() for
all portal strategies, including those pertaining to multiple queries.
The QueryDescs for strategies handled by PortalRunMulti() are
remembered in the Portal in a new List field 'qdescs', allocated in a
new memory context 'queryContext'. This new arrangment is to make it
easier to discard and recreate a Portal if the CachedPlan goes stale
during setup.
---
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 146 ++++++---
src/backend/commands/extension.c | 2 +
src/backend/commands/matview.c | 3 +-
src/backend/commands/portalcmds.c | 16 +-
src/backend/commands/prepare.c | 32 +-
src/backend/executor/execMain.c | 116 +++++--
src/backend/executor/execParallel.c | 9 +-
src/backend/executor/execPartition.c | 14 +
src/backend/executor/execProcnode.c | 50 ++-
src/backend/executor/execUtils.c | 65 +++-
src/backend/executor/functions.c | 2 +
src/backend/executor/nodeAgg.c | 6 +-
src/backend/executor/nodeAppend.c | 54 ++--
src/backend/executor/nodeBitmapAnd.c | 31 +-
src/backend/executor/nodeBitmapHeapscan.c | 9 +-
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 31 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 8 +-
src/backend/executor/nodeGather.c | 4 +-
src/backend/executor/nodeGatherMerge.c | 3 +-
src/backend/executor/nodeGroup.c | 7 +-
src/backend/executor/nodeHash.c | 10 +-
src/backend/executor/nodeHashjoin.c | 10 +-
src/backend/executor/nodeIncrementalSort.c | 7 +-
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 3 +-
src/backend/executor/nodeLockRows.c | 3 +-
src/backend/executor/nodeMaterial.c | 7 +-
src/backend/executor/nodeMemoize.c | 7 +-
src/backend/executor/nodeMergeAppend.c | 53 +--
src/backend/executor/nodeMergejoin.c | 10 +-
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/executor/nodeNestloop.c | 10 +-
src/backend/executor/nodeProjectSet.c | 7 +-
src/backend/executor/nodeRecursiveunion.c | 10 +-
src/backend/executor/nodeResult.c | 7 +-
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 4 +-
src/backend/executor/nodeSort.c | 7 +-
src/backend/executor/nodeSubqueryscan.c | 7 +-
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 4 +-
src/backend/executor/nodeWindowAgg.c | 6 +-
src/backend/executor/spi.c | 49 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++
src/backend/tcop/postgres.c | 13 +-
src/backend/tcop/pquery.c | 305 +++++++++---------
src/backend/utils/cache/lsyscache.c | 21 ++
src/backend/utils/cache/plancache.c | 149 +++------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/executor/execdesc.h | 5 +
src/include/executor/executor.h | 21 ++
src/include/nodes/execnodes.h | 6 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 +
src/include/utils/portal.h | 4 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 63 +++-
.../expected/cached-plan-replan.out | 156 +++++++++
.../specs/cached-plan-replan.spec | 61 ++++
69 files changed, 1218 insertions(+), 558 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..8244194681 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate,
* ExecutorStart computes a result tupdesc for us
*/
ExecutorStart(cstate->queryDesc, 0);
+ Assert(cstate->queryDesc->plan_valid);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 15f9bddcdf..91632d83e4 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,91 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to have been invalidated since its
+ * creation.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ ExecutorStart(queryDesc, eflags);
+ if (!queryDesc->plan_valid)
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +604,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +622,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4898,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 0eabe18335..5a76343123 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
ExecutorStart(qdesc, 0);
+ Assert(qdesc->plan_valid);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index f9a3bdfc3a..1c1ce1e17d 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
ExecutorStart(queryDesc, 0);
+ Assert(queryDesc->plan_valid);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..4abbec054b 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(portal->plan_valid);
/*
* We're done; the query won't actually be run until PerformPortalFetch is
@@ -249,6 +250,17 @@ PerformPortalClose(const char *name)
PortalDrop(portal, false);
}
+/*
+ * Release a portal's QueryDesc.
+ */
+void
+PortalQueryFinish(QueryDesc *queryDesc)
+{
+ ExecutorFinish(queryDesc);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+}
+
/*
* PortalCleanup
*
@@ -295,9 +307,7 @@ PortalCleanup(Portal portal)
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
- FreeQueryDesc(queryDesc);
+ PortalQueryFinish(queryDesc);
CurrentResourceOwner = saveResourceOwner;
}
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..c9070ed97f 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if portal->plan_valid is false which tells that the
+ * cached plan was found to have been invalidated when initializing one of
+ * the plan trees contained in it.
*/
PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
(void) PortalRun(portal, count, false, true, dest, dest, qc);
PortalDrop(portal, false);
@@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index c76fdf59ec..00db6eb307 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -611,6 +611,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by AcquirePlannerLocks() if a
+ * cached plan is being executed.
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -820,6 +830,23 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ *
+ * Normally, the plan tree given in queryDesc->plannedstmt is known to be
+ * valid in that *all* relations contained in plannedstmt->relationOids have
+ * already been locked. That may not be the case however if the plannedstmt
+ * comes from a CachedPlan, one given in queryDesc->cplan, not all relations
+ * referenced in the plan would have been locked; to wit AcquirePlannerLocks()
+ * only locks relations mentioned in the query but not any child relations
+ * that would have been added by the planner. Locks on the child relations
+ * will be taken when initializing their Scan nodes in ExecInitNode() that is
+ * done here. If the CachedPlan gets invalidated as these locks are taken,
+ * plan tree initialization is suspended at the point when such invalidation is
+ * first detected and queryDesc->planstat will be set to NULL and
+ * queryDesc->plan_valid to false. Callers must retry the execution after
+ * creating a new CachedPlan in that case, after properly releasing the
+ * resources of this QueryDesc, which includes calling ExecutorFinish() and
+ * ExecutorEnd() on the EState contained therein.
* ----------------------------------------------------------------
*/
static void
@@ -830,7 +857,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -841,10 +868,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -877,6 +905,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -920,6 +950,16 @@ InitPlan(QueryDesc *queryDesc, int eflags)
/* signal that this EState is not used for EPQ */
estate->es_epq_active = NULL;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query. Child relations that appear under Append/MergeAppend are
+ * locked in ExecInit[Merge]Append().
+ */
+ if (estate->es_cachedplan)
+ ExecLockElidedAppendChildRelations(estate,
+ plannedstmt->elidedAppendChildRels);
+
/*
* Initialize private state information for each SubPlan. We must do this
* before running ExecInitNode on the main query tree, since
@@ -944,10 +984,14 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
-
i++;
}
@@ -957,6 +1001,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -999,7 +1048,19 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ queryDesc->plan_valid = true;
+ return;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_inited_plannodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ queryDesc->plan_valid = false;
}
/*
@@ -1417,7 +1478,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked.
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -1495,18 +1556,15 @@ ExecEndPlan(PlanState *planstate, EState *estate)
ListCell *l;
/*
- * shut down the node-type-specific query processing
- */
- ExecEndNode(planstate);
-
- /*
- * for subplans too
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during InitPlan(), both in the main plan tree and those
+ * in subplans (es_subplanstates), if any.
*/
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/*
@@ -2849,7 +2907,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2936,6 +2995,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2979,6 +3044,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3001,6 +3072,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
@@ -3021,13 +3096,16 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during EvalPlanQualStart(), both in the main plan tree
+ * and those in subplans (es_subplanstates), if any.
+ */
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..72f1511720 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,14 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1431,6 +1437,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
ExecutorStart(queryDesc, fpes->eflags);
+ Assert(queryDesc->plan_valid);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..f3bb1d4591 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * Also, all non-NULL PlanState nodes are added to
+ * estate->es_inited_plannodes for ExecEndPlan() to iterate over to close
+ * each one using ExecEndNode().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
@@ -411,6 +428,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /*
+ * Remember valid PlanState nodes in EState for the processing in
+ * ExecEndPlan().
+ */
+ estate->es_inited_plannodes = lappend(estate->es_inited_plannodes,
+ result);
+
return result;
}
@@ -545,29 +569,21 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
+ * Child nodes, if any, would have been closed by the caller, so the
+ * ExecEnd* routine for a given node type is only responsible for
+ * cleaning up the resources local to that node.
+ *
+ * After this operation, the query plan containing this node will not be
+ * able to be processed any further. This should be called only after
* the query plan has been fully executed.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 4758ab4132..599db4d597 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -804,7 +804,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (!IsParallelWorker() &&
+ (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0)
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -820,9 +821,11 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
else
{
/*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
+ * Take a lock if we are a parallel worker or if the caller has set
+ * the GET_LOCKS flag (callers that open a child relation when
+ * initializing a cached plan). Parallel workers need to have
+ * their own local lock on the relation. This ensures sane
+ * behavior in case the parent process exits before we do.
*/
rel = table_open(rte->relid, rte->rellockmode);
}
@@ -833,6 +836,58 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockElidedAppendChildRelations
+ * Lock child relations whose parent Append/MergeAppend node was removed
+ * by the planner
+ */
+void
+ExecLockElidedAppendChildRelations(EState *estate, List *elidedAppendChildRels)
+{
+ ListCell *l;
+
+ foreach(l, elidedAppendChildRels)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(lfirst_int(l), estate);
+
+ Assert(rte->rtekind == RTE_RELATION);
+ Assert(OidIsValid(rte->relid));
+ Assert(rte->rellockmode != NoLock);
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+}
+
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * XXX - no need really to lock the first member of each bitmapset
+ * because it stands for the root parent mentioned in the query
+ * that should always have been locked before entering the
+ * executor.
+ */
+ i = -1;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -848,6 +903,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..c88f72bc4e 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
else
eflags = 0; /* default run-to-completion flags */
ExecutorStart(es->qd, eflags);
+ Assert(es->qd->plan_valid);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..54f742820b 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
@@ -4304,7 +4306,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4366,9 +4367,6 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..bd81d0ca4b 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,24 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +165,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -188,6 +208,13 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendplanstates = (PlanState **) palloc(nplans *
sizeof(PlanState *));
+ /*
+ * Set eflags so that ExecInitNode() recursively locks child relations
+ * appearing in appendplans.
+ */
+ if (estate->es_cachedplan)
+ estate->es_top_eflags |= EXEC_FLAG_GET_LOCKS;
+
/*
* call ExecInitNode on each of the valid plans to be executed and save
* the results into the appendplanstates array.
@@ -221,8 +248,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
+ estate->es_top_eflags &= ~EXEC_FLAG_GET_LOCKS;
+
appendstate->as_first_partial_plan = firstvalid;
appendstate->appendplans = appendplanstates;
appendstate->as_nplans = nplans;
@@ -376,30 +407,15 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /*
+ * Nothing to do as subscans of the append node would be cleaned up by
+ * ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..187aea4bb8 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -168,33 +169,15 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..ee1008519b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,11 +667,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
-
/*
* release bitmaps and buffers if any
*/
@@ -763,11 +758,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..3f51918fe1 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -186,33 +187,15 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..207165f44f 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
@@ -309,10 +313,6 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
-
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..400c8b42ed 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
@@ -248,7 +251,6 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..9077c4bc55 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
@@ -288,7 +290,6 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..976e739ab7 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -226,15 +228,10 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..fc7a6b2ccc 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
@@ -413,18 +415,10 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
-
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..4c4b39ce2d 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
@@ -878,12 +882,6 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
-
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 34257ce34b..8dfb2cb0f6 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -1101,11 +1103,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
-
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..1cc884bc65 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
@@ -535,7 +537,6 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..77731c0c8c 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
@@ -386,7 +388,6 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..a38b9805a5 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
@@ -250,11 +252,6 @@ ExecEndMaterial(MaterialState *node)
if (node->tuplestorestate != NULL)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..a8997ba7da 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
@@ -1099,11 +1101,6 @@ ExecEndMemoize(MemoizeState *node)
* free exprcontext
*/
ExecFreeExprContext(&node->ss.ps);
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..06a4827e00 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,24 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +113,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -140,6 +160,13 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.resultopsset = true;
mergestate->ps.resultopsfixed = false;
+ /*
+ * Set eflags so that ExecInitNode() recursively locks child relations
+ * appearing in appendplans.
+ */
+ if (estate->es_cachedplan)
+ estate->es_top_eflags |= EXEC_FLAG_GET_LOCKS;
+
/*
* call ExecInitNode on each of the valid plans to be executed and save
* the results into the mergeplanstates array.
@@ -151,8 +178,12 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
+ estate->es_top_eflags &= ~EXEC_FLAG_GET_LOCKS;
+
mergestate->ps.ps_ProjInfo = NULL;
/*
@@ -310,30 +341,14 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /*
+ * Nothing to do as subscans would be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..c6644c6816 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
@@ -1654,12 +1658,6 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
-
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..0c3aeb1154 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
@@ -4460,11 +4467,6 @@ ExecEndModifyTable(ModifyTableState *node)
* Terminate EPQ execution if active
*/
EvalPlanQualEnd(&node->mt_epqstate);
-
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..71a1f8101c 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
@@ -374,12 +378,6 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
-
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..abcbd7e765 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -329,11 +331,6 @@ ExecEndProjectSet(ProjectSetState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..84a706458a 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
@@ -280,12 +284,6 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
MemoryContextDelete(node->tempContext);
if (node->tableContext)
MemoryContextDelete(node->tableContext);
-
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..330ca68d12 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -249,11 +251,6 @@ ExecEndResult(ResultState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..912cf7b37f 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
@@ -589,8 +591,6 @@ ExecEndSetOp(SetOpState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..1ba53373c2 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -317,11 +319,6 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
-
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
}
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..12014250ae 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
@@ -178,11 +180,6 @@ ExecEndSubqueryScan(SubqueryScanState *node)
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..bd71033622 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
@@ -172,8 +174,6 @@ ExecEndUnique(UniqueState *node)
ExecClearTuple(node->ps.ps_ResultTupleSlot);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..483f23da18 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
@@ -2681,7 +2683,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2713,9 +2714,6 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
/* -----------------
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..07b1f453e2 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, paramLI, 0, snapshot);
@@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2668,14 +2680,32 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+ if (!qdesc->plan_valid)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2880,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2926,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 01b6cc1f7d..4931fb2da7 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1233,6 +1233,7 @@ exec_simple_query(const char *query_string)
* Start the portal. No parameters here.
*/
PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(portal->plan_valid);
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1738,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,10 +2030,19 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/*
* Apply the result format requests to the portal.
*/
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..09ee6069f9 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt)
* to be used for cursors).
*
* On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * tupdesc (if any) is known, unless portal->plan_valid is set to false, in
+ * which case, the caller must retry after generating a new CachedPlan.
*/
void
PortalStart(Portal portal, ParamListInfo params,
@@ -435,10 +353,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +419,52 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
ExecutorStart(queryDesc, myeflags);
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ PopActiveSnapshot();
+ portal->plan_valid = false;
+ goto early_exit;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -532,33 +472,11 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -578,11 +496,86 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot if we'll need to update
+ * its command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ portal->plan_valid = false;
+ goto early_exit;
+ }
+ }
+ }
+
portal->tupDesc = NULL;
+ portal->plan_valid = true;
break;
}
}
@@ -594,19 +587,18 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+early_exit:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
-
- portal->status = PORTAL_READY;
}
/*
@@ -1193,7 +1185,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1206,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1271,23 +1264,38 @@ PortalRunMulti(Portal portal,
else
UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1346,8 +1354,15 @@ PortalRunMulti(Portal portal,
* Increment command counter between queries, but not after the last
* one.
*/
- if (lnext(portal->stmts, stmtlist_item) != NULL)
+ if (lnext(portal->qdescs, qdesc_item) != NULL)
CommandCounterIncrement();
+
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 60978f9415..de3fc756e2 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 87210fcf62..16fb85312b 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -788,8 +788,14 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * Note though that if the plan contains any child relations that would have
+ * been added by the planner, which would not have been locked yet (because
+ * AcquirePlannerLocks() only locks relations that would be present in the
+ * range table before entering the planner), the plan could go stale before
+ * it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables, and if it is not,
+ * ask the caller to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -803,60 +809,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1126,8 +1128,15 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, that is, only the locks on the tables mentioned in the query have
+ * been taken. If any of those tables have inheritance/partition tables, the
+ * executor must also lock them before executing the plan and if the plan gets
+ * invalidated as a result of taking those locks, must ask the caller to get
+ * a new plan by calling here again. Locking of the child tables must be
+ * deferred to the executor like this, because not all child tables may need
+ * to be locked; some may get pruned during the executor plan initialization
+ * phase (InitPlan()).
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1360,8 +1369,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1735,58 +1744,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..c36c25b497 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -47,6 +50,7 @@ typedef struct QueryDesc
TupleDesc tupDesc; /* descriptor for result tuples */
EState *estate; /* executor's query-wide state */
PlanState *planstate; /* tree of per-plan-node state */
+ bool plan_valid; /* is planstate tree fully valid? */
/* This field is set by ExecutorRun */
bool already_executed; /* true if previously executed */
@@ -57,6 +61,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index ac02247947..7cba22acc4 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -61,6 +62,10 @@
* WITH_NO_DATA indicates that we are performing REFRESH MATERIALIZED VIEW
* ... WITH NO DATA. Currently, the only effect is to suppress errors about
* scanning unpopulated materialized views.
+ *
+ * GET_LOCKS indicates that the caller of ExecutorStart() is executing a
+ * cached plan which must be validated by taking the remaining locks necessary
+ * for execution.
*/
#define EXEC_FLAG_EXPLAIN_ONLY 0x0001 /* EXPLAIN, no ANALYZE */
#define EXEC_FLAG_EXPLAIN_GENERIC 0x0002 /* EXPLAIN (GENERIC_PLAN) */
@@ -69,6 +74,8 @@
#define EXEC_FLAG_MARK 0x0010 /* need mark/restore */
#define EXEC_FLAG_SKIP_TRIGGERS 0x0020 /* skip AfterTrigger setup */
#define EXEC_FLAG_WITH_NO_DATA 0x0040 /* REFRESH ... WITH NO DATA */
+#define EXEC_FLAG_GET_LOCKS 0x0400 /* should the executor lock
+ * relations? */
/* Hook for plugins to get control in ExecutorStart() */
@@ -256,6 +263,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +608,9 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockElidedAppendChildRelations(EState *estate,
+ List *elidedAppendChildRels);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..f0c5177b06 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
@@ -671,6 +673,10 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_inited_plannodes; /* List of PlanState of nodes from the
+ * plan tree that were fully
+ * initialized */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index 4f5418b972..3074e604dd 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index a443181d41..8990fe72e3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..24d420b9e9 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,9 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
+ bool plan_valid; /* are plans in qdescs ready for execution? */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
@@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal,
CommandTag commandTag,
List *stmts,
CachedPlan *cplan);
+extern void PortalQueryFinish(QueryDesc *queryDesc);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
extern void PortalHashTableDeleteAll(void);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..515b2c0c95 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static void
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ queryDesc->cplan->is_valid ? "valid" : "not valid");
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +123,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-03 13:27 Daniel Gustafsson <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Daniel Gustafsson @ 2023-07-03 13:27 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
> On 8 Jun 2023, at 16:23, Amit Langote <[email protected]> wrote:
>
> Here is a new version.
The local planstate variable in the hunk below is shadowing the function
parameter planstate which cause a compiler warning:
@@ -1495,18 +1556,15 @@ ExecEndPlan(PlanState *planstate, EState *estate)
ListCell *l;
/*
- * shut down the node-type-specific query processing
- */
- ExecEndNode(planstate);
-
- /*
- * for subplans too
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during InitPlan(), both in the main plan tree and those
+ * in subplans (es_subplanstates), if any.
*/
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-06 14:29 Amit Langote <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-07-06 14:29 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Mon, Jul 3, 2023 at 10:27 PM Daniel Gustafsson <[email protected]> wrote:
> > On 8 Jun 2023, at 16:23, Amit Langote <[email protected]> wrote:
> >
> > Here is a new version.
>
> The local planstate variable in the hunk below is shadowing the function
> parameter planstate which cause a compiler warning:
Thanks Daniel for the heads up.
Attached new version fixes that and contains a few other notable
changes. Before going into the details of those changes, let me
reiterate in broad strokes what the patch is trying to do.
The idea is to move the locking of some tables referenced in a cached
(generic) plan from plancache/GetCachedPlan() to the
executor/ExecutorStart(). Specifically, the locking of inheritance
child tables. Why? Because partition pruning with "initial pruning
steps" contained in the Append/MergeAppend nodes may eliminate some
child tables that need not have been locked to begin with, though the
pruning can only occur during ExecutorStart().
After applying this patch, GetCachedPlan() only locks the tables that
are directly mentioned in the query to ensure that the
analyzed-rewritten-but-unplanned query tree backing a given CachedPlan
is still valid (cf RevalidateCachedQuery()), but not the tables in the
CachedPlan that would have been added by the planner. Tables in a
CachePlan that would not be locked currently only include the
inheritance child tables / partitions of the tables mentioned in the
query. This means that the plan trees in a given CachedPlan returned
by GetCachedPlan() are only partially valid and are subject to
invalidation because concurrent sessions can possibly modify the child
tables referenced in them before ExecutorStart() gets around to
locking them. If the concurrent modifications do happen,
ExecutorStart() is now equipped to detect them by way of noticing that
the CachedPlan is invalidated and inform the caller to discard and
recreate the CachedPlan. This entails changing all the call sites of
ExecutorStart() that pass it a plan tree from a CachedPlan to
implement the replan-and-retry-execution loop.
Given the above, ExecutorStart(), which has not needed so far to take
any locks (except on indexes mentioned in IndexScans), now needs to
lock child tables if executing a cached plan which contains them. In
the previous versions, the patch used a flag passed in
EState.es_top_eflags to signal ExecGetRangeTableRelation() to lock the
table. The flag would be set in ExecInitAppend() and
ExecInitMergeAppend() for the duration of the loop that initializes
child subplans with the assumption that that's where the child tables
would be opened. But not all child subplans of Append/MergeAppend
scan child tables (think UNION ALL queries), so this approach can
result in redundant locking. Worse, I needed to invent
PlannedStmt.elidedAppendChildRelations to separately track child
tables whose Scan nodes' parent Append/MergeAppend would be removed by
setrefs.c in some cases.
So, this new patch uses a flag in the RangeTblEntry itself to denote
if the table is a child table instead of the above roundabout way.
ExecGetRangeTableRelation() can simply look at the RTE to decide
whether to take a lock or not. I considered adding a new bool field,
but noticed we already have inFromCl to track if a given RTE is for
table/entity directly mentioned in the query or for something added
behind-the-scenes into the range table as the field's description in
parsenodes.h says. RTEs for child tables are added behind-the-scenes
by the planner and it makes perfect sense to me to mark their inFromCl
as false. I can't find anything that relies on the current behavior
of inFromCl being set to the same value as the root inheritance parent
(true). Patch 0002 makes this change for child RTEs.
A few other notes:
* A parallel worker does ExecutorStart() without access to the
CachedPlan that the leader may have gotten its plan tree from. This
means that parallel workers do not have the ability to detect plan
tree invalidations. I think that's fine, because if the leader would
have been able to launch workers at all, it would also have gotten all
the locks to protect the (portion of) the plan tree that the workers
would be executing. I had an off-list discussion about this with
Robert and he mentioned his concern that each parallel worker would
have its own view of which child subplans of a parallel Append are
"valid" that depends on the result of its own evaluation of initial
pruning. So, there may be race conditions whereby a worker may try
to execute plan nodes that are no longer valid, for example, if the
partition a worker considers valid is not viewed as such by the leader
and thus not locked. I shared my thoughts as to why that sounds
unlikely at [1], though maybe I'm a bit too optimistic?
* For multi-query portals, you can't now do ExecutorStart()
immediately followed by ExecutorRun() for each query in the portal,
because ExecutorStart() may now fail to start a plan if it gets
invalidated. So PortalStart() now does ExecutorStart()s for all
queries and remembers the QueryDescs for PortalRun() then to do
ExecutorRun()s using. A consequence of this is that
CommandCounterIncrement() now must be done between the
ExecutorStart()s of the individual plans in PortalStart() and not
between the ExecutorRun()s in PortalRunMulti(). make check-world
passes with this new arrangement, though I'm not entirely confident
that there are no problems lurking.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
[1] https://postgr/es/m/CA+HiwqFA=swkzgGK8AmXUNFtLeEXFJwFyY3E7cTxvL46aa1OTw@mail.gmail.com
Attachments:
[application/octet-stream] v40-0004-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 2-v40-0004-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 7643c369e4877ad77d57c38e7c86c888efff3771 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v40 4/4] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a2f6ac9d1c..053d8a2dc2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1650,12 +1650,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index af92d2b3c3..f0320cfa34 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -837,6 +837,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f0c5177b06..be06c40766 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v40-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 3-v40-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From cede423bb7cfeb879b78f63087d18326cda13f5b Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v40 2/4] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 88b03cc472..c1360f87ee 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -995,11 +995,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v40-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch (125.3K, 4-v40-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 5fcc6f7b2d55efbd61dc4cf9ac69f3ff6b4f81a4 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v40 3/4] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, InitPlan().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during InitPlan() so that
initial pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Though some
plan tree subnodes may get fully initialized by ExecInitNode() before
the CachedPlan's invalidation is detected, so to ensure that they
are released by ExecEndPlan(), ExecInitNode() now adds the PlanState
nodes of the nodes that are fully initialized to a new List in
EState called es_inited_plannodes. ExecEndPlan() releases them
individually by calling ExecEndNode() on each element of the new
List. ExecEndNode() is no longer recursive, because all nodes that
need to be closed can be found in es_inited_plannodes.
Call sites that use GetCachedPlan() to get the plan trees to pass to
the executor should now be prepared to handle the case where the old
CachedPlan gets invalidated during ExecutorStart() as described
above. So this commit refactors the relevant code sites to move the
ExecutorStart() call closer to the GetCachedPlan() to implement the
replan loop conveniently.
Given this new behavior, PortalStart() now must always perform
ExecutorStart() to be able to drop and recreate cached plans if
needed, which is currently only done so for single-query portals.
For multi-query portals, the QueryDescs that are now created during
PortalStart() are remembered in a new List field of Portal called
'qdescs' and allocated in a new memory context 'queryContext'.
PortalRunMulti() now simply performs ExecutorRun() on the
QueryDescs found in 'qdescs'.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 2 +
src/backend/commands/matview.c | 3 +-
src/backend/commands/portalcmds.c | 16 +-
src/backend/commands/prepare.c | 32 +-
src/backend/executor/execMain.c | 106 +++++-
src/backend/executor/execParallel.c | 12 +-
src/backend/executor/execPartition.c | 14 +
src/backend/executor/execProcnode.c | 50 ++-
src/backend/executor/execUtils.c | 63 +++-
src/backend/executor/functions.c | 2 +
src/backend/executor/nodeAgg.c | 6 +-
src/backend/executor/nodeAppend.c | 48 ++-
src/backend/executor/nodeBitmapAnd.c | 31 +-
src/backend/executor/nodeBitmapHeapscan.c | 9 +-
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 31 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 8 +-
src/backend/executor/nodeGather.c | 4 +-
src/backend/executor/nodeGatherMerge.c | 3 +-
src/backend/executor/nodeGroup.c | 7 +-
src/backend/executor/nodeHash.c | 10 +-
src/backend/executor/nodeHashjoin.c | 10 +-
src/backend/executor/nodeIncrementalSort.c | 7 +-
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 3 +-
src/backend/executor/nodeLockRows.c | 3 +-
src/backend/executor/nodeMaterial.c | 7 +-
src/backend/executor/nodeMemoize.c | 7 +-
src/backend/executor/nodeMergeAppend.c | 47 ++-
src/backend/executor/nodeMergejoin.c | 10 +-
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/executor/nodeNestloop.c | 10 +-
src/backend/executor/nodeProjectSet.c | 7 +-
src/backend/executor/nodeRecursiveunion.c | 10 +-
src/backend/executor/nodeResult.c | 7 +-
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 4 +-
src/backend/executor/nodeSort.c | 7 +-
src/backend/executor/nodeSubqueryscan.c | 7 +-
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 4 +-
src/backend/executor/nodeWindowAgg.c | 6 +-
src/backend/executor/spi.c | 49 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++
src/backend/tcop/postgres.c | 13 +-
src/backend/tcop/pquery.c | 340 +++++++++---------
src/backend/utils/cache/lsyscache.c | 21 ++
src/backend/utils/cache/plancache.c | 149 +++-----
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/executor/execdesc.h | 5 +
src/include/executor/executor.h | 13 +
src/include/nodes/execnodes.h | 6 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 +
src/include/utils/portal.h | 4 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 63 +++-
.../expected/cached-plan-replan.out | 156 ++++++++
.../specs/cached-plan-replan.spec | 61 ++++
69 files changed, 1193 insertions(+), 588 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..8244194681 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate,
* ExecutorStart computes a result tupdesc for us
*/
ExecutorStart(cstate->queryDesc, 0);
+ Assert(cstate->queryDesc->plan_valid);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..b1ea45ef2c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ ExecutorStart(queryDesc, eflags);
+ if (!queryDesc->plan_valid)
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4897,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 0eabe18335..5a76343123 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
ExecutorStart(qdesc, 0);
+ Assert(qdesc->plan_valid);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index f9a3bdfc3a..1c1ce1e17d 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
ExecutorStart(queryDesc, 0);
+ Assert(queryDesc->plan_valid);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..4abbec054b 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(portal->plan_valid);
/*
* We're done; the query won't actually be run until PerformPortalFetch is
@@ -249,6 +250,17 @@ PerformPortalClose(const char *name)
PortalDrop(portal, false);
}
+/*
+ * Release a portal's QueryDesc.
+ */
+void
+PortalQueryFinish(QueryDesc *queryDesc)
+{
+ ExecutorFinish(queryDesc);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+}
+
/*
* PortalCleanup
*
@@ -295,9 +307,7 @@ PortalCleanup(Portal portal)
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
- FreeQueryDesc(queryDesc);
+ PortalQueryFinish(queryDesc);
CurrentResourceOwner = saveResourceOwner;
}
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..c9070ed97f 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if portal->plan_valid is false which tells that the
+ * cached plan was found to have been invalidated when initializing one of
+ * the plan trees contained in it.
*/
PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
(void) PortalRun(portal, count, false, true, dest, dest, qc);
PortalDrop(portal, false);
@@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..a2f6ac9d1c 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -620,6 +620,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,6 +840,23 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ *
+ * Normally, the plan tree given in queryDesc->plannedstmt is known to be
+ * valid in a race-free manner, that is, all relations contained in
+ * plannedstmt->relationOids would have already been locked. That is not the
+ * case however if the plannedstmt comes from a CachedPlan, one given in
+ * queryDesc->cplan. That's because GetCachedPlan() only locks the tables
+ * that are mentioned in the original query but not the child tables, which
+ * would have been added to the plan by the planner. In that case, locks on
+ * child tables will be taken when initializing their Scan nodes in
+ * ExecInitNode() to be done here. If the CachedPlan gets invalidated as
+ * those locks are taken, plan tree initialization is suspended at the point
+ * where the invalidation is first detected, queryDesc->planstate will be set
+ * to NULL, and queryDesc->plan_valid to false. Callers must retry the
+ * execution after creating a new CachedPlan in that case, after properly
+ * releasing the resources of this QueryDesc, which includes calling
+ * ExecutorFinish() and ExecutorEnd() on the EState contained therein.
* ----------------------------------------------------------------
*/
static void
@@ -839,7 +867,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +878,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +915,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +984,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +1002,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1049,19 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ queryDesc->plan_valid = true;
+ return;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_inited_plannodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ queryDesc->plan_valid = false;
}
/*
@@ -1426,7 +1479,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -1504,18 +1557,15 @@ ExecEndPlan(PlanState *planstate, EState *estate)
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during InitPlan(), both in the main plan tree and those
+ * in subplans (es_subplanstates), if any.
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -2858,7 +2908,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2945,6 +2996,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2988,6 +3045,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3010,6 +3073,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
@@ -3030,13 +3097,16 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during EvalPlanQualStart(), both in the main plan tree
+ * and those in subplans (es_subplanstates), if any.
+ */
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..42df7b6428 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1431,6 +1440,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
ExecutorStart(queryDesc, fpes->eflags);
+ Assert(queryDesc->plan_valid);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..f3bb1d4591 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * Also, all non-NULL PlanState nodes are added to
+ * estate->es_inited_plannodes for ExecEndPlan() to iterate over to close
+ * each one using ExecEndNode().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
@@ -411,6 +428,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /*
+ * Remember valid PlanState nodes in EState for the processing in
+ * ExecEndPlan().
+ */
+ estate->es_inited_plannodes = lappend(estate->es_inited_plannodes,
+ result);
+
return result;
}
@@ -545,29 +569,21 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
+ * Child nodes, if any, would have been closed by the caller, so the
+ * ExecEnd* routine for a given node type is only responsible for
+ * cleaning up the resources local to that node.
+ *
+ * After this operation, the query plan containing this node will not be
+ * able to be processed any further. This should be called only after
* the query plan has been fully executed.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..af92d2b3c3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -804,7 +804,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -817,15 +835,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -833,6 +842,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -848,6 +889,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..c88f72bc4e 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
else
eflags = 0; /* default run-to-completion flags */
ExecutorStart(es->qd, eflags);
+ Assert(es->qd->plan_valid);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..54f742820b 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
@@ -4304,7 +4306,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4366,9 +4367,6 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a6dadb7d07 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,27 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +168,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +244,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
@@ -376,30 +401,15 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /*
+ * Nothing to do as subscans of the append node would be cleaned up by
+ * ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..187aea4bb8 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -168,33 +169,15 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..ee1008519b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,11 +667,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
-
/*
* release bitmaps and buffers if any
*/
@@ -763,11 +758,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..3f51918fe1 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -186,33 +187,15 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..207165f44f 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
@@ -309,10 +313,6 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
-
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..400c8b42ed 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
@@ -248,7 +251,6 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..9077c4bc55 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
@@ -288,7 +290,6 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..976e739ab7 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -226,15 +228,10 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..fc7a6b2ccc 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
@@ -413,18 +415,10 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
-
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..4c4b39ce2d 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
@@ -878,12 +882,6 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
-
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..5b11afeb96 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -1101,11 +1103,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
-
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..1cc884bc65 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
@@ -535,7 +537,6 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..77731c0c8c 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
@@ -386,7 +388,6 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..a38b9805a5 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
@@ -250,11 +252,6 @@ ExecEndMaterial(MaterialState *node)
if (node->tuplestorestate != NULL)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..a8997ba7da 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
@@ -1099,11 +1101,6 @@ ExecEndMemoize(MemoizeState *node)
* free exprcontext
*/
ExecFreeExprContext(&node->ss.ps);
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8718f20825 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,27 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +116,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +174,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
@@ -310,30 +335,14 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /*
+ * Nothing to do as subscans would be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..c6644c6816 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
@@ -1654,12 +1658,6 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
-
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..0c3aeb1154 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
@@ -4460,11 +4467,6 @@ ExecEndModifyTable(ModifyTableState *node)
* Terminate EPQ execution if active
*/
EvalPlanQualEnd(&node->mt_epqstate);
-
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..71a1f8101c 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
@@ -374,12 +378,6 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
-
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..abcbd7e765 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -329,11 +331,6 @@ ExecEndProjectSet(ProjectSetState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..84a706458a 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
@@ -280,12 +284,6 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
MemoryContextDelete(node->tempContext);
if (node->tableContext)
MemoryContextDelete(node->tableContext);
-
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..330ca68d12 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -249,11 +251,6 @@ ExecEndResult(ResultState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..912cf7b37f 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
@@ -589,8 +591,6 @@ ExecEndSetOp(SetOpState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..1ba53373c2 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -317,11 +319,6 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
-
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
}
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..12014250ae 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
@@ -178,11 +180,6 @@ ExecEndSubqueryScan(SubqueryScanState *node)
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..bd71033622 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
@@ -172,8 +174,6 @@ ExecEndUnique(UniqueState *node)
ExecClearTuple(node->ps.ps_ResultTupleSlot);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..483f23da18 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
@@ -2681,7 +2683,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2713,9 +2714,6 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
/* -----------------
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..07b1f453e2 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, paramLI, 0, snapshot);
@@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2668,14 +2680,32 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+ if (!qdesc->plan_valid)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2880,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2926,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 01b6cc1f7d..4931fb2da7 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1233,6 +1233,7 @@ exec_simple_query(const char *query_string)
* Start the portal. No parameters here.
*/
PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(portal->plan_valid);
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1738,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,10 +2030,19 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/*
* Apply the result format requests to the portal.
*/
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..dab971ab0f 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt)
* to be used for cursors).
*
* On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * tupdesc (if any) is known, unless portal->plan_valid is set to false, in
+ * which case, the caller must retry after generating a new CachedPlan.
*/
void
PortalStart(Portal portal, ParamListInfo params,
@@ -435,10 +353,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +419,52 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
ExecutorStart(queryDesc, myeflags);
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ PopActiveSnapshot();
+ portal->plan_valid = false;
+ goto early_exit;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -532,33 +472,11 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -578,11 +496,87 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ portal->plan_valid = false;
+ goto early_exit;
+ }
+ }
+ }
+
portal->tupDesc = NULL;
+ portal->plan_valid = true;
break;
}
}
@@ -594,19 +588,18 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+early_exit:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
-
- portal->status = PORTAL_READY;
}
/*
@@ -1193,7 +1186,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1207,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1227,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1255,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1342,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 60978f9415..de3fc756e2 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 3d3f7a9bea..e6237d70b3 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,14 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * Note though that if the plan contains any child relations that would have
+ * been added by the planner, which would not have been locked yet (because
+ * AcquirePlannerLocks() only locks relations that would be present in the
+ * range table before entering the planner), the plan could go stale before
+ * it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables, and if it is not,
+ * ask the caller to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +811,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1130,15 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, that is, only the locks on the tables mentioned in the query have
+ * been taken. If any of those tables have inheritance/partition tables, the
+ * executor must also lock them before executing the plan and if the plan gets
+ * invalidated as a result of taking those locks, must ask the caller to get
+ * a new plan by calling here again. Locking of the child tables must be
+ * deferred to the executor like this, because not all child tables may need
+ * to be locked; some may get pruned during the executor plan initialization
+ * phase (InitPlan()).
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1362,8 +1371,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1737,58 +1746,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..c36c25b497 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -47,6 +50,7 @@ typedef struct QueryDesc
TupleDesc tupDesc; /* descriptor for result tuples */
EState *estate; /* executor's query-wide state */
PlanState *planstate; /* tree of per-plan-node state */
+ bool plan_valid; /* is planstate tree fully valid? */
/* This field is set by ExecutorRun */
bool already_executed; /* true if previously executed */
@@ -57,6 +61,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index ac02247947..640b905973 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..f0c5177b06 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
@@ -671,6 +673,10 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_inited_plannodes; /* List of PlanState of nodes from the
+ * plan tree that were fully
+ * initialized */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index 4f5418b972..3074e604dd 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index a443181d41..8990fe72e3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..24d420b9e9 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,9 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
+ bool plan_valid; /* are plans in qdescs ready for execution? */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
@@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal,
CommandTag commandTag,
List *stmts,
CachedPlan *cplan);
+extern void PortalQueryFinish(QueryDesc *queryDesc);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
extern void PortalHashTableDeleteAll(void);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..515b2c0c95 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static void
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ queryDesc->cplan->is_valid ? "valid" : "not valid");
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +123,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v40-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 5-v40-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 54554c16763831037b48d7c7686f883c2c882108 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v40 1/4] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ec73789bc2..5ee2b5b7f9 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0e12fdeb60..7bc6eec364 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7835,8 +7835,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index c63758cb2b..396c83e357 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1754,6 +1754,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1830,6 +1832,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-13 12:58 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-07-13 12:58 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; +Cc: Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Thu, Jul 6, 2023 at 11:29 PM Amit Langote <[email protected]> wrote:
> On Mon, Jul 3, 2023 at 10:27 PM Daniel Gustafsson <[email protected]> wrote:
> > > On 8 Jun 2023, at 16:23, Amit Langote <[email protected]> wrote:
> > > Here is a new version.
> >
> > The local planstate variable in the hunk below is shadowing the function
> > parameter planstate which cause a compiler warning:
>
> Thanks Daniel for the heads up.
>
> Attached new version fixes that and contains a few other notable
> changes. Before going into the details of those changes, let me
> reiterate in broad strokes what the patch is trying to do.
>
> The idea is to move the locking of some tables referenced in a cached
> (generic) plan from plancache/GetCachedPlan() to the
> executor/ExecutorStart(). Specifically, the locking of inheritance
> child tables. Why? Because partition pruning with "initial pruning
> steps" contained in the Append/MergeAppend nodes may eliminate some
> child tables that need not have been locked to begin with, though the
> pruning can only occur during ExecutorStart().
>
> After applying this patch, GetCachedPlan() only locks the tables that
> are directly mentioned in the query to ensure that the
> analyzed-rewritten-but-unplanned query tree backing a given CachedPlan
> is still valid (cf RevalidateCachedQuery()), but not the tables in the
> CachedPlan that would have been added by the planner. Tables in a
> CachePlan that would not be locked currently only include the
> inheritance child tables / partitions of the tables mentioned in the
> query. This means that the plan trees in a given CachedPlan returned
> by GetCachedPlan() are only partially valid and are subject to
> invalidation because concurrent sessions can possibly modify the child
> tables referenced in them before ExecutorStart() gets around to
> locking them. If the concurrent modifications do happen,
> ExecutorStart() is now equipped to detect them by way of noticing that
> the CachedPlan is invalidated and inform the caller to discard and
> recreate the CachedPlan. This entails changing all the call sites of
> ExecutorStart() that pass it a plan tree from a CachedPlan to
> implement the replan-and-retry-execution loop.
>
> Given the above, ExecutorStart(), which has not needed so far to take
> any locks (except on indexes mentioned in IndexScans), now needs to
> lock child tables if executing a cached plan which contains them. In
> the previous versions, the patch used a flag passed in
> EState.es_top_eflags to signal ExecGetRangeTableRelation() to lock the
> table. The flag would be set in ExecInitAppend() and
> ExecInitMergeAppend() for the duration of the loop that initializes
> child subplans with the assumption that that's where the child tables
> would be opened. But not all child subplans of Append/MergeAppend
> scan child tables (think UNION ALL queries), so this approach can
> result in redundant locking. Worse, I needed to invent
> PlannedStmt.elidedAppendChildRelations to separately track child
> tables whose Scan nodes' parent Append/MergeAppend would be removed by
> setrefs.c in some cases.
>
> So, this new patch uses a flag in the RangeTblEntry itself to denote
> if the table is a child table instead of the above roundabout way.
> ExecGetRangeTableRelation() can simply look at the RTE to decide
> whether to take a lock or not. I considered adding a new bool field,
> but noticed we already have inFromCl to track if a given RTE is for
> table/entity directly mentioned in the query or for something added
> behind-the-scenes into the range table as the field's description in
> parsenodes.h says. RTEs for child tables are added behind-the-scenes
> by the planner and it makes perfect sense to me to mark their inFromCl
> as false. I can't find anything that relies on the current behavior
> of inFromCl being set to the same value as the root inheritance parent
> (true). Patch 0002 makes this change for child RTEs.
>
> A few other notes:
>
> * A parallel worker does ExecutorStart() without access to the
> CachedPlan that the leader may have gotten its plan tree from. This
> means that parallel workers do not have the ability to detect plan
> tree invalidations. I think that's fine, because if the leader would
> have been able to launch workers at all, it would also have gotten all
> the locks to protect the (portion of) the plan tree that the workers
> would be executing. I had an off-list discussion about this with
> Robert and he mentioned his concern that each parallel worker would
> have its own view of which child subplans of a parallel Append are
> "valid" that depends on the result of its own evaluation of initial
> pruning. So, there may be race conditions whereby a worker may try
> to execute plan nodes that are no longer valid, for example, if the
> partition a worker considers valid is not viewed as such by the leader
> and thus not locked. I shared my thoughts as to why that sounds
> unlikely at [1], though maybe I'm a bit too optimistic?
>
> * For multi-query portals, you can't now do ExecutorStart()
> immediately followed by ExecutorRun() for each query in the portal,
> because ExecutorStart() may now fail to start a plan if it gets
> invalidated. So PortalStart() now does ExecutorStart()s for all
> queries and remembers the QueryDescs for PortalRun() then to do
> ExecutorRun()s using. A consequence of this is that
> CommandCounterIncrement() now must be done between the
> ExecutorStart()s of the individual plans in PortalStart() and not
> between the ExecutorRun()s in PortalRunMulti(). make check-world
> passes with this new arrangement, though I'm not entirely confident
> that there are no problems lurking.
In an absolutely brown-paper-bag moment, I realized that I had not
updated src/backend/executor/README to reflect the changes to the
executor's control flow that this patch makes. That is, after
scrapping the old design back in January whose details *were*
reflected in the patches before that redesign.
Anyway, the attached fixes that.
Tom, do you think you have bandwidth in the near future to give this
another look? I think I've addressed the comments that you had given
back in April, though as mentioned in the previous message, there may
still be some funny-looking aspects still remaining. In any case, I
have no intention of pressing ahead with the patch without another
committer having had a chance to sign off on it.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/x-patch] v41-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 2-v41-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 91dca92a9918335952178af1f3827087d3b33485 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v41 1/4] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ec73789bc2..5ee2b5b7f9 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0e12fdeb60..7bc6eec364 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7835,8 +7835,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index c63758cb2b..396c83e357 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1754,6 +1754,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1830,6 +1832,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/x-patch] v41-0004-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 3-v41-0004-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From b8fd387987d28b13a2229515d0fa451e332b9d8e Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v41 4/4] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a2f6ac9d1c..053d8a2dc2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1650,12 +1650,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index af92d2b3c3..f0320cfa34 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -837,6 +837,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f0c5177b06..be06c40766 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/x-patch] v41-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 4-v41-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From 02f7fa054649fda593cd17fc2c0cb40543cf9bae Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v41 2/4] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index efb5c3e098..a891272e6e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/x-patch] v41-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch (128.6K, 5-v41-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 17f8ce26b86c3e8aebfd5279fc6bbdc7997894f8 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v41 3/4] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, InitPlan().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during InitPlan() so that
initial pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Though some
plan tree subnodes may get fully initialized by ExecInitNode() before
the CachedPlan's invalidation is detected, so to ensure that they
are released by ExecEndPlan(), ExecInitNode() now adds the PlanState
nodes of the nodes that are fully initialized to a new List in
EState called es_inited_plannodes. ExecEndPlan() releases them
individually by calling ExecEndNode() on each element of the new
List. ExecEndNode() is no longer recursive, because all nodes that
need to be closed can be found in es_inited_plannodes.
Call sites that use GetCachedPlan() to get the plan trees to pass to
the executor should now be prepared to handle the case where the old
CachedPlan gets invalidated during ExecutorStart() as described
above. So this commit refactors the relevant code sites to move the
ExecutorStart() call closer to the GetCachedPlan() to implement the
replan loop conveniently.
Given this new behavior, PortalStart() now must always perform
ExecutorStart() to be able to drop and recreate cached plans if
needed, which is currently only done so for single-query portals.
For multi-query portals, the QueryDescs that are now created during
PortalStart() are remembered in a new List field of Portal called
'qdescs' and allocated in a new memory context 'queryContext'.
PortalRunMulti() now simply performs ExecutorRun() on the
QueryDescs found in 'qdescs'.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 2 +
src/backend/commands/matview.c | 3 +-
src/backend/commands/portalcmds.c | 16 +-
src/backend/commands/prepare.c | 32 +-
src/backend/executor/README | 39 +-
src/backend/executor/execMain.c | 106 +++++-
src/backend/executor/execParallel.c | 12 +-
src/backend/executor/execPartition.c | 14 +
src/backend/executor/execProcnode.c | 50 ++-
src/backend/executor/execUtils.c | 63 +++-
src/backend/executor/functions.c | 2 +
src/backend/executor/nodeAgg.c | 6 +-
src/backend/executor/nodeAppend.c | 48 ++-
src/backend/executor/nodeBitmapAnd.c | 31 +-
src/backend/executor/nodeBitmapHeapscan.c | 9 +-
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 31 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 8 +-
src/backend/executor/nodeGather.c | 4 +-
src/backend/executor/nodeGatherMerge.c | 3 +-
src/backend/executor/nodeGroup.c | 7 +-
src/backend/executor/nodeHash.c | 10 +-
src/backend/executor/nodeHashjoin.c | 10 +-
src/backend/executor/nodeIncrementalSort.c | 7 +-
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 3 +-
src/backend/executor/nodeLockRows.c | 3 +-
src/backend/executor/nodeMaterial.c | 7 +-
src/backend/executor/nodeMemoize.c | 7 +-
src/backend/executor/nodeMergeAppend.c | 47 ++-
src/backend/executor/nodeMergejoin.c | 10 +-
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/executor/nodeNestloop.c | 10 +-
src/backend/executor/nodeProjectSet.c | 7 +-
src/backend/executor/nodeRecursiveunion.c | 10 +-
src/backend/executor/nodeResult.c | 7 +-
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 4 +-
src/backend/executor/nodeSort.c | 7 +-
src/backend/executor/nodeSubqueryscan.c | 7 +-
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 4 +-
src/backend/executor/nodeWindowAgg.c | 6 +-
src/backend/executor/spi.c | 49 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++
src/backend/tcop/postgres.c | 13 +-
src/backend/tcop/pquery.c | 340 +++++++++---------
src/backend/utils/cache/lsyscache.c | 21 ++
src/backend/utils/cache/plancache.c | 149 +++-----
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/executor/execdesc.h | 5 +
src/include/executor/executor.h | 13 +
src/include/nodes/execnodes.h | 6 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 +
src/include/utils/portal.h | 4 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 63 +++-
.../expected/cached-plan-replan.out | 156 ++++++++
.../specs/cached-plan-replan.spec | 61 ++++
70 files changed, 1231 insertions(+), 589 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..8244194681 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate,
* ExecutorStart computes a result tupdesc for us
*/
ExecutorStart(cstate->queryDesc, 0);
+ Assert(cstate->queryDesc->plan_valid);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..b1ea45ef2c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ ExecutorStart(queryDesc, eflags);
+ if (!queryDesc->plan_valid)
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4897,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 9a2ee1c600..72d60d9c4f 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
ExecutorStart(qdesc, 0);
+ Assert(qdesc->plan_valid);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..a64880b719 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
ExecutorStart(queryDesc, 0);
+ Assert(queryDesc->plan_valid);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..4abbec054b 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(portal->plan_valid);
/*
* We're done; the query won't actually be run until PerformPortalFetch is
@@ -249,6 +250,17 @@ PerformPortalClose(const char *name)
PortalDrop(portal, false);
}
+/*
+ * Release a portal's QueryDesc.
+ */
+void
+PortalQueryFinish(QueryDesc *queryDesc)
+{
+ ExecutorFinish(queryDesc);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+}
+
/*
* PortalCleanup
*
@@ -295,9 +307,7 @@ PortalCleanup(Portal portal)
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
- FreeQueryDesc(queryDesc);
+ PortalQueryFinish(queryDesc);
CurrentResourceOwner = saveResourceOwner;
}
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..c9070ed97f 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if portal->plan_valid is false which tells that the
+ * cached plan was found to have been invalidated when initializing one of
+ * the plan trees contained in it.
*/
PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
(void) PortalRun(portal, count, false, true, dest, dest, qc);
PortalDrop(portal, false);
@@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..110062e0e8 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,38 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables, which are
+added to the query's range table during planning, if they are present in a
+cached plan tree would not have been locked.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to executor-initialization-time pruning. So the
+locking of child tables must wait till execution-initialization-time, which
+occurs during ExecInitNode() on the plan nodes containing the child tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree (after executor-initialization-pruning, if any,
+has been performed), which it does by looking at CachedPlan.is_valid of the
+CachedPlan passed to it. If the plan tree is indeed stale (is_valid=false),
+the executor must give up continuing to intiatialize it any further and
+return to the caller letting it know that the execution must be retried with
+a new plan tree.
+
Query Processing Control Flow
-----------------------------
@@ -300,6 +332,11 @@ This is a sketch of control flow for full query processing:
creates per-tuple context
ExecInitExpr
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() shown above after
+taking a lock on a child table, the control is returned to the caller, which
+must redo the steps from CreateQueryDesc with a new plan tree.
+
ExecutorRun
ExecProcNode --- recursively called in per-query context
ExecEvalExpr --- called in per-tuple context
@@ -310,7 +347,7 @@ This is a sketch of control flow for full query processing:
AfterTriggerEndQuery
ExecutorEnd
- ExecEndNode --- recursively releases resources
+ ExecEndNode --- releases resources
FreeExecutorState
frees per-query context and child contexts
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..a2f6ac9d1c 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -620,6 +620,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,6 +840,23 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ *
+ * Normally, the plan tree given in queryDesc->plannedstmt is known to be
+ * valid in a race-free manner, that is, all relations contained in
+ * plannedstmt->relationOids would have already been locked. That is not the
+ * case however if the plannedstmt comes from a CachedPlan, one given in
+ * queryDesc->cplan. That's because GetCachedPlan() only locks the tables
+ * that are mentioned in the original query but not the child tables, which
+ * would have been added to the plan by the planner. In that case, locks on
+ * child tables will be taken when initializing their Scan nodes in
+ * ExecInitNode() to be done here. If the CachedPlan gets invalidated as
+ * those locks are taken, plan tree initialization is suspended at the point
+ * where the invalidation is first detected, queryDesc->planstate will be set
+ * to NULL, and queryDesc->plan_valid to false. Callers must retry the
+ * execution after creating a new CachedPlan in that case, after properly
+ * releasing the resources of this QueryDesc, which includes calling
+ * ExecutorFinish() and ExecutorEnd() on the EState contained therein.
* ----------------------------------------------------------------
*/
static void
@@ -839,7 +867,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +878,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +915,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +984,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +1002,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1049,19 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ queryDesc->plan_valid = true;
+ return;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_inited_plannodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ queryDesc->plan_valid = false;
}
/*
@@ -1426,7 +1479,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -1504,18 +1557,15 @@ ExecEndPlan(PlanState *planstate, EState *estate)
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during InitPlan(), both in the main plan tree and those
+ * in subplans (es_subplanstates), if any.
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -2858,7 +2908,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2945,6 +2996,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2988,6 +3045,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3010,6 +3073,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
@@ -3030,13 +3097,16 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during EvalPlanQualStart(), both in the main plan tree
+ * and those in subplans (es_subplanstates), if any.
+ */
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..42df7b6428 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1431,6 +1440,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
ExecutorStart(queryDesc, fpes->eflags);
+ Assert(queryDesc->plan_valid);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..f3bb1d4591 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * Also, all non-NULL PlanState nodes are added to
+ * estate->es_inited_plannodes for ExecEndPlan() to iterate over to close
+ * each one using ExecEndNode().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
@@ -411,6 +428,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /*
+ * Remember valid PlanState nodes in EState for the processing in
+ * ExecEndPlan().
+ */
+ estate->es_inited_plannodes = lappend(estate->es_inited_plannodes,
+ result);
+
return result;
}
@@ -545,29 +569,21 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
+ * Child nodes, if any, would have been closed by the caller, so the
+ * ExecEnd* routine for a given node type is only responsible for
+ * cleaning up the resources local to that node.
+ *
+ * After this operation, the query plan containing this node will not be
+ * able to be processed any further. This should be called only after
* the query plan has been fully executed.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..af92d2b3c3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -804,7 +804,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -817,15 +835,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -833,6 +842,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -848,6 +889,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..c88f72bc4e 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
else
eflags = 0; /* default run-to-completion flags */
ExecutorStart(es->qd, eflags);
+ Assert(es->qd->plan_valid);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..54f742820b 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
@@ -4304,7 +4306,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4366,9 +4367,6 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a6dadb7d07 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,27 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +168,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +244,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
@@ -376,30 +401,15 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /*
+ * Nothing to do as subscans of the append node would be cleaned up by
+ * ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..187aea4bb8 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -168,33 +169,15 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..ee1008519b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,11 +667,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
-
/*
* release bitmaps and buffers if any
*/
@@ -763,11 +758,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..3f51918fe1 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -186,33 +187,15 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..207165f44f 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
@@ -309,10 +313,6 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
-
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..400c8b42ed 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
@@ -248,7 +251,6 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..9077c4bc55 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
@@ -288,7 +290,6 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..976e739ab7 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -226,15 +228,10 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..fc7a6b2ccc 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
@@ -413,18 +415,10 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
-
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..4c4b39ce2d 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
@@ -878,12 +882,6 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
-
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..5b11afeb96 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -1101,11 +1103,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
-
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..1cc884bc65 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
@@ -535,7 +537,6 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..77731c0c8c 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
@@ -386,7 +388,6 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..a38b9805a5 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
@@ -250,11 +252,6 @@ ExecEndMaterial(MaterialState *node)
if (node->tuplestorestate != NULL)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..a8997ba7da 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
@@ -1099,11 +1101,6 @@ ExecEndMemoize(MemoizeState *node)
* free exprcontext
*/
ExecFreeExprContext(&node->ss.ps);
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8718f20825 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,27 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +116,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +174,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
@@ -310,30 +335,14 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /*
+ * Nothing to do as subscans would be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..c6644c6816 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
@@ -1654,12 +1658,6 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
-
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..0c3aeb1154 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
@@ -4460,11 +4467,6 @@ ExecEndModifyTable(ModifyTableState *node)
* Terminate EPQ execution if active
*/
EvalPlanQualEnd(&node->mt_epqstate);
-
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..71a1f8101c 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
@@ -374,12 +378,6 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
-
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..abcbd7e765 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -329,11 +331,6 @@ ExecEndProjectSet(ProjectSetState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..84a706458a 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
@@ -280,12 +284,6 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
MemoryContextDelete(node->tempContext);
if (node->tableContext)
MemoryContextDelete(node->tableContext);
-
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..330ca68d12 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -249,11 +251,6 @@ ExecEndResult(ResultState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..912cf7b37f 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
@@ -589,8 +591,6 @@ ExecEndSetOp(SetOpState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..1ba53373c2 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -317,11 +319,6 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
-
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
}
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..12014250ae 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
@@ -178,11 +180,6 @@ ExecEndSubqueryScan(SubqueryScanState *node)
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..bd71033622 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
@@ -172,8 +174,6 @@ ExecEndUnique(UniqueState *node)
ExecClearTuple(node->ps.ps_ResultTupleSlot);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..483f23da18 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
@@ -2681,7 +2683,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2713,9 +2714,6 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
/* -----------------
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..07b1f453e2 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, paramLI, 0, snapshot);
@@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2668,14 +2680,32 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+ if (!qdesc->plan_valid)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2880,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2926,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..160aef92f8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1233,6 +1233,7 @@ exec_simple_query(const char *query_string)
* Start the portal. No parameters here.
*/
PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(portal->plan_valid);
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1738,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,10 +2030,19 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/*
* Apply the result format requests to the portal.
*/
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..dab971ab0f 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt)
* to be used for cursors).
*
* On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * tupdesc (if any) is known, unless portal->plan_valid is set to false, in
+ * which case, the caller must retry after generating a new CachedPlan.
*/
void
PortalStart(Portal portal, ParamListInfo params,
@@ -435,10 +353,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +419,52 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
ExecutorStart(queryDesc, myeflags);
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ PopActiveSnapshot();
+ portal->plan_valid = false;
+ goto early_exit;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -532,33 +472,11 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -578,11 +496,87 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ portal->plan_valid = false;
+ goto early_exit;
+ }
+ }
+ }
+
portal->tupDesc = NULL;
+ portal->plan_valid = true;
break;
}
}
@@ -594,19 +588,18 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+early_exit:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
-
- portal->status = PORTAL_READY;
}
/*
@@ -1193,7 +1186,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1207,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1227,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1255,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1342,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 60978f9415..de3fc756e2 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 3d3f7a9bea..e6237d70b3 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,14 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * Note though that if the plan contains any child relations that would have
+ * been added by the planner, which would not have been locked yet (because
+ * AcquirePlannerLocks() only locks relations that would be present in the
+ * range table before entering the planner), the plan could go stale before
+ * it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables, and if it is not,
+ * ask the caller to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +811,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1130,15 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, that is, only the locks on the tables mentioned in the query have
+ * been taken. If any of those tables have inheritance/partition tables, the
+ * executor must also lock them before executing the plan and if the plan gets
+ * invalidated as a result of taking those locks, must ask the caller to get
+ * a new plan by calling here again. Locking of the child tables must be
+ * deferred to the executor like this, because not all child tables may need
+ * to be locked; some may get pruned during the executor plan initialization
+ * phase (InitPlan()).
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1362,8 +1371,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1737,58 +1746,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..c36c25b497 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -47,6 +50,7 @@ typedef struct QueryDesc
TupleDesc tupDesc; /* descriptor for result tuples */
EState *estate; /* executor's query-wide state */
PlanState *planstate; /* tree of per-plan-node state */
+ bool plan_valid; /* is planstate tree fully valid? */
/* This field is set by ExecutorRun */
bool already_executed; /* true if previously executed */
@@ -57,6 +61,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index ac02247947..640b905973 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..f0c5177b06 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
@@ -671,6 +673,10 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_inited_plannodes; /* List of PlanState of nodes from the
+ * plan tree that were fully
+ * initialized */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index 4f5418b972..3074e604dd 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index a443181d41..8990fe72e3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..24d420b9e9 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,9 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
+ bool plan_valid; /* are plans in qdescs ready for execution? */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
@@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal,
CommandTag commandTag,
List *stmts,
CachedPlan *cplan);
+extern void PortalQueryFinish(QueryDesc *queryDesc);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
extern void PortalHashTableDeleteAll(void);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..515b2c0c95 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static void
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ queryDesc->cplan->is_valid ? "valid" : "not valid");
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +123,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-17 16:32 Thom Brown <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Thom Brown @ 2023-07-17 16:32 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Thu, 13 Jul 2023 at 13:59, Amit Langote <[email protected]> wrote:
> In an absolutely brown-paper-bag moment, I realized that I had not
> updated src/backend/executor/README to reflect the changes to the
> executor's control flow that this patch makes. That is, after
> scrapping the old design back in January whose details *were*
> reflected in the patches before that redesign.
>
> Anyway, the attached fixes that.
>
> Tom, do you think you have bandwidth in the near future to give this
> another look? I think I've addressed the comments that you had given
> back in April, though as mentioned in the previous message, there may
> still be some funny-looking aspects still remaining. In any case, I
> have no intention of pressing ahead with the patch without another
> committer having had a chance to sign off on it.
I've only just started taking a look at this, and my first test drive
yields very impressive results:
8192 partitions (3 runs, 10000 rows)
Head 391.294989 382.622481 379.252236
Patched 13088.145995 13406.135531 13431.828051
Looking at your changes to README, I would like to suggest rewording
the following:
+table during planning. This means that inheritance child tables, which are
+added to the query's range table during planning, if they are present in a
+cached plan tree would not have been locked.
To:
This means that inheritance child tables present in a cached plan
tree, which are added to the query's range table during planning,
would not have been locked.
Also, further down:
s/intiatialize/initialize/
I'll carry on taking a closer look and see if I can break it.
Thom
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-18 07:26 Amit Langote <[email protected]>
parent: Thom Brown <[email protected]>
0 siblings, 2 replies; 31+ messages in thread
From: Amit Langote @ 2023-07-18 07:26 UTC (permalink / raw)
To: Thom Brown <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
Hi Thom,
On Tue, Jul 18, 2023 at 1:33 AM Thom Brown <[email protected]> wrote:
> On Thu, 13 Jul 2023 at 13:59, Amit Langote <[email protected]> wrote:
> > In an absolutely brown-paper-bag moment, I realized that I had not
> > updated src/backend/executor/README to reflect the changes to the
> > executor's control flow that this patch makes. That is, after
> > scrapping the old design back in January whose details *were*
> > reflected in the patches before that redesign.
> >
> > Anyway, the attached fixes that.
> >
> > Tom, do you think you have bandwidth in the near future to give this
> > another look? I think I've addressed the comments that you had given
> > back in April, though as mentioned in the previous message, there may
> > still be some funny-looking aspects still remaining. In any case, I
> > have no intention of pressing ahead with the patch without another
> > committer having had a chance to sign off on it.
>
> I've only just started taking a look at this, and my first test drive
> yields very impressive results:
>
> 8192 partitions (3 runs, 10000 rows)
> Head 391.294989 382.622481 379.252236
> Patched 13088.145995 13406.135531 13431.828051
Just to be sure, did you use pgbench --Mprepared with plan_cache_mode
= force_generic_plan in postgresql.conf?
> Looking at your changes to README, I would like to suggest rewording
> the following:
>
> +table during planning. This means that inheritance child tables, which are
> +added to the query's range table during planning, if they are present in a
> +cached plan tree would not have been locked.
>
> To:
>
> This means that inheritance child tables present in a cached plan
> tree, which are added to the query's range table during planning,
> would not have been locked.
>
> Also, further down:
>
> s/intiatialize/initialize/
>
> I'll carry on taking a closer look and see if I can break it.
Thanks for looking. I've fixed these issues in the attached updated
patch. I've also changed the position of a newly added paragraph in
src/backend/executor/README so that it doesn't break the flow of the
existing text.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v42-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 2-v42-0001-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From b6413e324c5be9273a3c33aa026c06cdfb710da7 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v42 1/4] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..8ac1d3909b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v42-0004-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 3-v42-0004-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From b9b750fd6323342672613c7efcdb47287264baa4 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v42 4/4] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a2f6ac9d1c..053d8a2dc2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1650,12 +1650,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index af92d2b3c3..f0320cfa34 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -837,6 +837,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f0c5177b06..be06c40766 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v42-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 4-v42-0002-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From a16ff85f49bd4d67413b7397fe2ad2fd642f0284 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v42 2/4] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index efb5c3e098..a891272e6e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v42-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch (128.6K, 5-v42-0003-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 731074be2036f02d081fe7c731e21330b6b47b4a Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v42 3/4] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, InitPlan().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during InitPlan() so that
initial pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Though some
plan tree subnodes may get fully initialized by ExecInitNode() before
the CachedPlan's invalidation is detected, so to ensure that they
are released by ExecEndPlan(), ExecInitNode() now adds the PlanState
nodes of the nodes that are fully initialized to a new List in
EState called es_inited_plannodes. ExecEndPlan() releases them
individually by calling ExecEndNode() on each element of the new
List. ExecEndNode() is no longer recursive, because all nodes that
need to be closed can be found in es_inited_plannodes.
Call sites that use GetCachedPlan() to get the plan trees to pass to
the executor should now be prepared to handle the case where the old
CachedPlan gets invalidated during ExecutorStart() as described
above. So this commit refactors the relevant code sites to move the
ExecutorStart() call closer to the GetCachedPlan() to implement the
replan loop conveniently.
Given this new behavior, PortalStart() now must always perform
ExecutorStart() to be able to drop and recreate cached plans if
needed, which is currently only done so for single-query portals.
For multi-query portals, the QueryDescs that are now created during
PortalStart() are remembered in a new List field of Portal called
'qdescs' and allocated in a new memory context 'queryContext'.
PortalRunMulti() now simply performs ExecutorRun() on the
QueryDescs found in 'qdescs'.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 2 +
src/backend/commands/matview.c | 3 +-
src/backend/commands/portalcmds.c | 16 +-
src/backend/commands/prepare.c | 32 +-
src/backend/executor/README | 41 ++-
src/backend/executor/execMain.c | 106 +++++-
src/backend/executor/execParallel.c | 12 +-
src/backend/executor/execPartition.c | 14 +
src/backend/executor/execProcnode.c | 50 ++-
src/backend/executor/execUtils.c | 63 +++-
src/backend/executor/functions.c | 2 +
src/backend/executor/nodeAgg.c | 6 +-
src/backend/executor/nodeAppend.c | 48 ++-
src/backend/executor/nodeBitmapAnd.c | 31 +-
src/backend/executor/nodeBitmapHeapscan.c | 9 +-
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 31 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 8 +-
src/backend/executor/nodeGather.c | 4 +-
src/backend/executor/nodeGatherMerge.c | 3 +-
src/backend/executor/nodeGroup.c | 7 +-
src/backend/executor/nodeHash.c | 10 +-
src/backend/executor/nodeHashjoin.c | 10 +-
src/backend/executor/nodeIncrementalSort.c | 7 +-
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 3 +-
src/backend/executor/nodeLockRows.c | 3 +-
src/backend/executor/nodeMaterial.c | 7 +-
src/backend/executor/nodeMemoize.c | 7 +-
src/backend/executor/nodeMergeAppend.c | 47 ++-
src/backend/executor/nodeMergejoin.c | 10 +-
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/executor/nodeNestloop.c | 10 +-
src/backend/executor/nodeProjectSet.c | 7 +-
src/backend/executor/nodeRecursiveunion.c | 10 +-
src/backend/executor/nodeResult.c | 7 +-
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 4 +-
src/backend/executor/nodeSort.c | 7 +-
src/backend/executor/nodeSubqueryscan.c | 7 +-
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 4 +-
src/backend/executor/nodeWindowAgg.c | 6 +-
src/backend/executor/spi.c | 49 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++
src/backend/tcop/postgres.c | 13 +-
src/backend/tcop/pquery.c | 340 +++++++++---------
src/backend/utils/cache/lsyscache.c | 21 ++
src/backend/utils/cache/plancache.c | 149 +++-----
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/executor/execdesc.h | 5 +
src/include/executor/executor.h | 13 +
src/include/nodes/execnodes.h | 6 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 +
src/include/utils/portal.h | 4 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 63 +++-
.../expected/cached-plan-replan.out | 156 ++++++++
.../specs/cached-plan-replan.spec | 61 ++++
70 files changed, 1233 insertions(+), 589 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..8244194681 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate,
* ExecutorStart computes a result tupdesc for us
*/
ExecutorStart(cstate->queryDesc, 0);
+ Assert(cstate->queryDesc->plan_valid);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..b1ea45ef2c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ ExecutorStart(queryDesc, eflags);
+ if (!queryDesc->plan_valid)
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4897,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 9a2ee1c600..72d60d9c4f 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
ExecutorStart(qdesc, 0);
+ Assert(qdesc->plan_valid);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..a64880b719 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
ExecutorStart(queryDesc, 0);
+ Assert(queryDesc->plan_valid);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..4abbec054b 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(portal->plan_valid);
/*
* We're done; the query won't actually be run until PerformPortalFetch is
@@ -249,6 +250,17 @@ PerformPortalClose(const char *name)
PortalDrop(portal, false);
}
+/*
+ * Release a portal's QueryDesc.
+ */
+void
+PortalQueryFinish(QueryDesc *queryDesc)
+{
+ ExecutorFinish(queryDesc);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+}
+
/*
* PortalCleanup
*
@@ -295,9 +307,7 @@ PortalCleanup(Portal portal)
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
- FreeQueryDesc(queryDesc);
+ PortalQueryFinish(queryDesc);
CurrentResourceOwner = saveResourceOwner;
}
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..c9070ed97f 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if portal->plan_valid is false which tells that the
+ * cached plan was found to have been invalidated when initializing one of
+ * the plan trees contained in it.
*/
PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
(void) PortalRun(portal, count, false, true, dest, dest, qc);
PortalDrop(portal, false);
@@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..fcbe266f8a 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,39 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to executor-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree (after executor-initialization-pruning, if any,
+has been performed), which it does by looking at CachedPlan.is_valid of the
+CachedPlan passed to it. If the plan tree is indeed stale (is_valid=false),
+the executor must give up continuing to initialize it any further and return
+to the caller letting it know that the execution must be retried with a new
+plan tree.
+
Query Processing Control Flow
-----------------------------
@@ -310,12 +343,18 @@ This is a sketch of control flow for full query processing:
AfterTriggerEndQuery
ExecutorEnd
- ExecEndNode --- recursively releases resources
+ ExecEndNode --- releases resources
FreeExecutorState
frees per-query context and child contexts
FreeQueryDesc
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
Per above comments, it's not really critical for ExecEndNode to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..a2f6ac9d1c 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -620,6 +620,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,6 +840,23 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ *
+ * Normally, the plan tree given in queryDesc->plannedstmt is known to be
+ * valid in a race-free manner, that is, all relations contained in
+ * plannedstmt->relationOids would have already been locked. That is not the
+ * case however if the plannedstmt comes from a CachedPlan, one given in
+ * queryDesc->cplan. That's because GetCachedPlan() only locks the tables
+ * that are mentioned in the original query but not the child tables, which
+ * would have been added to the plan by the planner. In that case, locks on
+ * child tables will be taken when initializing their Scan nodes in
+ * ExecInitNode() to be done here. If the CachedPlan gets invalidated as
+ * those locks are taken, plan tree initialization is suspended at the point
+ * where the invalidation is first detected, queryDesc->planstate will be set
+ * to NULL, and queryDesc->plan_valid to false. Callers must retry the
+ * execution after creating a new CachedPlan in that case, after properly
+ * releasing the resources of this QueryDesc, which includes calling
+ * ExecutorFinish() and ExecutorEnd() on the EState contained therein.
* ----------------------------------------------------------------
*/
static void
@@ -839,7 +867,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +878,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +915,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +984,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +1002,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1049,19 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ queryDesc->plan_valid = true;
+ return;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_inited_plannodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ queryDesc->plan_valid = false;
}
/*
@@ -1426,7 +1479,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -1504,18 +1557,15 @@ ExecEndPlan(PlanState *planstate, EState *estate)
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during InitPlan(), both in the main plan tree and those
+ * in subplans (es_subplanstates), if any.
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -2858,7 +2908,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2945,6 +2996,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2988,6 +3045,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3010,6 +3073,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
@@ -3030,13 +3097,16 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized during EvalPlanQualStart(), both in the main plan tree
+ * and those in subplans (es_subplanstates), if any.
+ */
+ foreach(l, estate->es_inited_plannodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..42df7b6428 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1431,6 +1440,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
ExecutorStart(queryDesc, fpes->eflags);
+ Assert(queryDesc->plan_valid);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..f3bb1d4591 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * Also, all non-NULL PlanState nodes are added to
+ * estate->es_inited_plannodes for ExecEndPlan() to iterate over to close
+ * each one using ExecEndNode().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
@@ -411,6 +428,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /*
+ * Remember valid PlanState nodes in EState for the processing in
+ * ExecEndPlan().
+ */
+ estate->es_inited_plannodes = lappend(estate->es_inited_plannodes,
+ result);
+
return result;
}
@@ -545,29 +569,21 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
+ * Child nodes, if any, would have been closed by the caller, so the
+ * ExecEnd* routine for a given node type is only responsible for
+ * cleaning up the resources local to that node.
+ *
+ * After this operation, the query plan containing this node will not be
+ * able to be processed any further. This should be called only after
* the query plan has been fully executed.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..af92d2b3c3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -804,7 +804,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -817,15 +835,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -833,6 +842,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -848,6 +889,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..c88f72bc4e 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
else
eflags = 0; /* default run-to-completion flags */
ExecutorStart(es->qd, eflags);
+ Assert(es->qd->plan_valid);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..54f742820b 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
@@ -4304,7 +4306,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4366,9 +4367,6 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a6dadb7d07 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,27 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +168,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +244,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
@@ -376,30 +401,15 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /*
+ * Nothing to do as subscans of the append node would be cleaned up by
+ * ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..187aea4bb8 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -168,33 +169,15 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..ee1008519b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,11 +667,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
-
/*
* release bitmaps and buffers if any
*/
@@ -763,11 +758,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..3f51918fe1 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
@@ -186,33 +187,15 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /*
+ * Nothing to do as any subscans that would have been initialized would
+ * be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..207165f44f 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
@@ -309,10 +313,6 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
-
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..400c8b42ed 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
@@ -248,7 +251,6 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..9077c4bc55 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
@@ -288,7 +290,6 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..976e739ab7 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -226,15 +228,10 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..fc7a6b2ccc 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
@@ -413,18 +415,10 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
-
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..4c4b39ce2d 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
@@ -878,12 +882,6 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
-
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..5b11afeb96 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -1101,11 +1103,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
-
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..1cc884bc65 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
@@ -535,7 +537,6 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..77731c0c8c 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
@@ -386,7 +388,6 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..a38b9805a5 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
@@ -250,11 +252,6 @@ ExecEndMaterial(MaterialState *node)
if (node->tuplestorestate != NULL)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..a8997ba7da 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
@@ -1099,11 +1101,6 @@ ExecEndMemoize(MemoizeState *node)
* free exprcontext
*/
ExecFreeExprContext(&node->ss.ps);
-
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8718f20825 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,27 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +116,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +174,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
@@ -310,30 +335,14 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /*
+ * Nothing to do as subscans would be cleaned up by ExecEndPlan().
+ */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..c6644c6816 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
@@ -1654,12 +1658,6 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
-
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..0c3aeb1154 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
@@ -4460,11 +4467,6 @@ ExecEndModifyTable(ModifyTableState *node)
* Terminate EPQ execution if active
*/
EvalPlanQualEnd(&node->mt_epqstate);
-
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..71a1f8101c 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
@@ -374,12 +378,6 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
-
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
}
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..abcbd7e765 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -329,11 +331,6 @@ ExecEndProjectSet(ProjectSetState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..84a706458a 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
@@ -280,12 +284,6 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
MemoryContextDelete(node->tempContext);
if (node->tableContext)
MemoryContextDelete(node->tableContext);
-
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..330ca68d12 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
@@ -249,11 +251,6 @@ ExecEndResult(ResultState *node)
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..912cf7b37f 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
@@ -589,8 +591,6 @@ ExecEndSetOp(SetOpState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..1ba53373c2 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
@@ -317,11 +319,6 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
-
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
}
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..12014250ae 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
@@ -178,11 +180,6 @@ ExecEndSubqueryScan(SubqueryScanState *node)
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..bd71033622 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
@@ -172,8 +174,6 @@ ExecEndUnique(UniqueState *node)
ExecClearTuple(node->ps.ps_ResultTupleSlot);
ExecFreeExprContext(&node->ps);
-
- ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..483f23da18 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
@@ -2681,7 +2683,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2713,9 +2714,6 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
}
/* -----------------
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..07b1f453e2 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, paramLI, 0, snapshot);
@@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2668,14 +2680,32 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+ if (!qdesc->plan_valid)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2880,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2926,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..160aef92f8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1233,6 +1233,7 @@ exec_simple_query(const char *query_string)
* Start the portal. No parameters here.
*/
PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(portal->plan_valid);
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1738,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,10 +2030,19 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!portal->plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/*
* Apply the result format requests to the portal.
*/
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..dab971ab0f 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt)
* to be used for cursors).
*
* On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * tupdesc (if any) is known, unless portal->plan_valid is set to false, in
+ * which case, the caller must retry after generating a new CachedPlan.
*/
void
PortalStart(Portal portal, ParamListInfo params,
@@ -435,10 +353,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +419,52 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
ExecutorStart(queryDesc, myeflags);
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ PopActiveSnapshot();
+ portal->plan_valid = false;
+ goto early_exit;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -532,33 +472,11 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -578,11 +496,87 @@ PortalStart(Portal portal, ParamListInfo params,
portal->atStart = true;
portal->atEnd = false; /* allow fetches */
portal->portalPos = 0;
+ portal->plan_valid = true;
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ if (!queryDesc->plan_valid)
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ portal->plan_valid = false;
+ goto early_exit;
+ }
+ }
+ }
+
portal->tupDesc = NULL;
+ portal->plan_valid = true;
break;
}
}
@@ -594,19 +588,18 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+early_exit:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
-
- portal->status = PORTAL_READY;
}
/*
@@ -1193,7 +1186,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1207,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1227,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1255,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1342,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 2c01a86c29..2adb1588a9 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 3d3f7a9bea..e6237d70b3 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,14 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * Note though that if the plan contains any child relations that would have
+ * been added by the planner, which would not have been locked yet (because
+ * AcquirePlannerLocks() only locks relations that would be present in the
+ * range table before entering the planner), the plan could go stale before
+ * it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables, and if it is not,
+ * ask the caller to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +811,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1130,15 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, that is, only the locks on the tables mentioned in the query have
+ * been taken. If any of those tables have inheritance/partition tables, the
+ * executor must also lock them before executing the plan and if the plan gets
+ * invalidated as a result of taking those locks, must ask the caller to get
+ * a new plan by calling here again. Locking of the child tables must be
+ * deferred to the executor like this, because not all child tables may need
+ * to be locked; some may get pruned during the executor plan initialization
+ * phase (InitPlan()).
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1362,8 +1371,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1737,58 +1746,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..c36c25b497 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -47,6 +50,7 @@ typedef struct QueryDesc
TupleDesc tupDesc; /* descriptor for result tuples */
EState *estate; /* executor's query-wide state */
PlanState *planstate; /* tree of per-plan-node state */
+ bool plan_valid; /* is planstate tree fully valid? */
/* This field is set by ExecutorRun */
bool already_executed; /* true if previously executed */
@@ -57,6 +61,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..e05e23bb4a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..f0c5177b06 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
@@ -671,6 +673,10 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_inited_plannodes; /* List of PlanState of nodes from the
+ * plan tree that were fully
+ * initialized */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index a443181d41..8990fe72e3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..24d420b9e9 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,9 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
+ bool plan_valid; /* are plans in qdescs ready for execution? */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
@@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal,
CommandTag commandTag,
List *stmts,
CachedPlan *cplan);
+extern void PortalQueryFinish(QueryDesc *queryDesc);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
extern void PortalHashTableDeleteAll(void);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..515b2c0c95 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static void
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ queryDesc->cplan->is_valid ? "valid" : "not valid");
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +123,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-07-18 08:36 Thom Brown <[email protected]>
parent: Amit Langote <[email protected]>
1 sibling, 0 replies; 31+ messages in thread
From: Thom Brown @ 2023-07-18 08:36 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Tue, 18 Jul 2023, 08:26 Amit Langote, <[email protected]> wrote:
> Hi Thom,
>
> On Tue, Jul 18, 2023 at 1:33 AM Thom Brown <[email protected]> wrote:
> > On Thu, 13 Jul 2023 at 13:59, Amit Langote <[email protected]>
> wrote:
> > > In an absolutely brown-paper-bag moment, I realized that I had not
> > > updated src/backend/executor/README to reflect the changes to the
> > > executor's control flow that this patch makes. That is, after
> > > scrapping the old design back in January whose details *were*
> > > reflected in the patches before that redesign.
> > >
> > > Anyway, the attached fixes that.
> > >
> > > Tom, do you think you have bandwidth in the near future to give this
> > > another look? I think I've addressed the comments that you had given
> > > back in April, though as mentioned in the previous message, there may
> > > still be some funny-looking aspects still remaining. In any case, I
> > > have no intention of pressing ahead with the patch without another
> > > committer having had a chance to sign off on it.
> >
> > I've only just started taking a look at this, and my first test drive
> > yields very impressive results:
> >
> > 8192 partitions (3 runs, 10000 rows)
> > Head 391.294989 382.622481 379.252236
> > Patched 13088.145995 13406.135531 13431.828051
>
> Just to be sure, did you use pgbench --Mprepared with plan_cache_mode
> = force_generic_plan in postgresql.conf?
>
I did.
For full disclosure, I also had max_locks_per_transaction set to 10000.
>
> > Looking at your changes to README, I would like to suggest rewording
> > the following:
> >
> > +table during planning. This means that inheritance child tables, which
> are
> > +added to the query's range table during planning, if they are present
> in a
> > +cached plan tree would not have been locked.
> >
> > To:
> >
> > This means that inheritance child tables present in a cached plan
> > tree, which are added to the query's range table during planning,
> > would not have been locked.
> >
> > Also, further down:
> >
> > s/intiatialize/initialize/
> >
> > I'll carry on taking a closer look and see if I can break it.
>
> Thanks for looking. I've fixed these issues in the attached updated
> patch. I've also changed the position of a newly added paragraph in
> src/backend/executor/README so that it doesn't break the flow of the
> existing text.
>
Thanks.
Thom
>
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-02 13:39 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-08-02 13:39 UTC (permalink / raw)
To: Thom Brown <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
While chatting with Robert about this patch set, he suggested that it
would be better to break out some executor refactoring changes from
the main patch (0003) into a separate patch. To wit, the changes to
make the PlanState tree cleanup in ExecEndPlan() non-recursive by
walking a flat list of PlanState nodes instead of the recursive tree
walk that ExecEndNode() currently does. That allows us to cleanly
handle the cases where the PlanState tree is only partially
constructed when ExecInitNode() detects in the middle of its
construction that the plan tree is no longer valid after receiving and
processing an invalidation message on locking child tables. Or at
least more cleanly than the previously proposed approach of adjusting
ExecEndNode() subroutines for the individual node types to gracefully
handle such partially initialized PlanState trees.
With the new approach, node type specific subroutines of ExecEndNode()
need not close its child nodes, because ExecEndPlan() would close each
node that would have been initialized directly. I couldn't find any
instance of breakage by this decoupling of child node cleanup from
their parent node's cleanup. Comments in ExecEndGather() and
ExecEndGatherMerge() appear to suggest that outerPlan must be closed
before the local cleanup:
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
But I don't think there's a problem, because what ExecShutdownGather()
does seems entirely independent of cleanup of outerPlan.
As for the performance impact of initializing the list of initialized
nodes to use during the cleanup phase, I couldn't find a regression,
nor any improvement by replacing the tree walk by linear scan of a
list. Actually, ExecEndNode() is pretty far down in the perf profile
anyway, so the performance difference caused by the patch hardly
matters. See the following contrived example:
create table f();
analyze f;
explain (costs off) select count(*) from f f1, f f2, f f3, f f4, f f5,
f f6, f f7, f f8, f f9, f f10;
QUERY PLAN
------------------------------------------------------------------------------
Aggregate
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Nested Loop
-> Seq Scan on f f1
-> Seq Scan on f f2
-> Seq Scan on f f3
-> Seq Scan on f f4
-> Seq Scan on f f5
-> Seq Scan on f f6
-> Seq Scan on f f7
-> Seq Scan on f f8
-> Seq Scan on f f9
-> Seq Scan on f f10
(20 rows)
do $$
begin
for i in 1..100000 loop
perform count(*) from f f1, f f2, f f3, f f4, f f5, f f6, f f7, f f8,
f f9, f f10;
end loop;
end; $$;
Times for the DO:
Unpatched:
Time: 756.353 ms
Time: 745.752 ms
Time: 749.184 ms
Patched:
Time: 737.717 ms
Time: 747.815 ms
Time: 753.456 ms
I've attached the new refactoring patch as 0001.
Another change I've made in the main patch is to change the API of
ExecutorStart() (and ExecutorStart_hook) more explicitly to return a
boolean indicating whether or not the plan initialization was
successful. That way seems better than making the callers figure that
out by seeing that QueryDesc.planstate is NULL and/or checking
QueryDesc.plan_valid. Correspondingly, PortalStart() now also returns
true or false matching what ExecutorStart() returned. I suppose this
better alerts any extensions that use the ExecutorStart_hook to fix
their code to do the right thing.
Having extracted the ExecEndNode() change, I'm also starting to feel
inclined to extract a couple of other bits from the main patch as
separate patches, such as moving the ExecutorStart() call from
PortalRun() to PortalStart() for the multi-query portals. I'll do
that in the next version.
Attachments:
[application/octet-stream] v43-0002-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 2-v43-0002-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 94117d1144355b2af718570244438853146edb4c Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v43 2/5] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..8ac1d3909b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v43-0001-Make-PlanState-tree-cleanup-non-recursive.patch (28.2K, 3-v43-0001-Make-PlanState-tree-cleanup-non-recursive.patch)
download | inline diff:
From a55bd363690bc4c28047e4b874ce80384e37c49d Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 1 Aug 2023 11:36:24 +0900
Subject: [PATCH v43 1/5] Make PlanState tree cleanup non-recursive
With this change, node type specific subroutines of ExecEndNode()
are no longer required to also clean up the child nodes of a given
node, only its own stuff. Instead, ExecEndPlan() calls
ExecInitNode() directly for each node in the PlanState tree by
iterating over a list (EState.es_planstate_nodes) of all those nodes
built during the ExecInitNode() traversal of the tree.
This changes the order in which the nodes get cleaned up, because
they are now cleaned up in the order in which they are added into
the list which is from leaf-level up to the root, whereas with the
current recursive approach cleanup occurs from the root to the
leaves. The change seems harmless though, because there isn't
necessarily any coupling between of the cleanup actions of parent
and child nodes.
The main motivation behind this change is to allow the cases in
the future where ExecInitNode() traversal of the plan tree may
be aborted in the middle resulting in a partially initialized
PlanState tree. Dealing with that case by making the cleanup
phase walk over a list of successfully initialized nodes seems
better / more robust than making the individual ExecEndNode()
subroutines deal with partially valid PlanState nodes.
---
src/backend/executor/README | 4 +-
src/backend/executor/execMain.c | 36 +++++++-------
src/backend/executor/execProcnode.c | 56 ++++++++++------------
src/backend/executor/execUtils.c | 2 +
src/backend/executor/nodeAgg.c | 4 +-
src/backend/executor/nodeAppend.c | 20 +-------
src/backend/executor/nodeBitmapAnd.c | 23 +--------
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeBitmapOr.c | 23 +--------
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeGather.c | 2 +-
src/backend/executor/nodeGatherMerge.c | 2 +-
src/backend/executor/nodeGroup.c | 5 +-
src/backend/executor/nodeHash.c | 8 +---
src/backend/executor/nodeHashjoin.c | 6 +--
src/backend/executor/nodeIncrementalSort.c | 5 +-
src/backend/executor/nodeLimit.c | 2 +-
src/backend/executor/nodeLockRows.c | 2 +-
src/backend/executor/nodeMaterial.c | 5 +-
src/backend/executor/nodeMemoize.c | 5 +-
src/backend/executor/nodeMergeAppend.c | 20 +-------
src/backend/executor/nodeMergejoin.c | 6 +--
src/backend/executor/nodeModifyTable.c | 7 +--
src/backend/executor/nodeNestloop.c | 6 +--
src/backend/executor/nodeProjectSet.c | 5 +-
src/backend/executor/nodeRecursiveunion.c | 6 +--
src/backend/executor/nodeResult.c | 5 +-
src/backend/executor/nodeSetOp.c | 2 +-
src/backend/executor/nodeSort.c | 5 +-
src/backend/executor/nodeSubqueryscan.c | 5 +-
src/backend/executor/nodeUnique.c | 2 +-
src/backend/executor/nodeWindowAgg.c | 4 +-
src/include/nodes/execnodes.h | 2 +
33 files changed, 80 insertions(+), 214 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..67a5c1769b 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -310,13 +310,13 @@ This is a sketch of control flow for full query processing:
AfterTriggerEndQuery
ExecutorEnd
- ExecEndNode --- recursively releases resources
+ ExecEndPlan --- releases plan resources
FreeExecutorState
frees per-query context and child contexts
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..235bb52ccc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -82,7 +82,7 @@ ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
-static void ExecEndPlan(PlanState *planstate, EState *estate);
+static void ExecEndPlan(EState *estate);
static void ExecutePlan(EState *estate, PlanState *planstate,
bool use_parallel_mode,
CmdType operation,
@@ -500,7 +500,7 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
*/
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndPlan(queryDesc->planstate, estate);
+ ExecEndPlan(estate);
/* do away with our snapshots */
UnregisterSnapshot(estate->es_snapshot);
@@ -1499,23 +1499,21 @@ ExecPostprocessPlan(EState *estate)
* ----------------------------------------------------------------
*/
static void
-ExecEndPlan(PlanState *planstate, EState *estate)
+ExecEndPlan(EState *estate)
{
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (es_plannedstmt->planTree) and those in subplans
+ * (es_plannedstmt->subplans).
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -3030,13 +3028,17 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (epqstate->plan) and those in subplans
+ * (es_plannedstmt->subplans).
+ */
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..653f74cf58 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -1,11 +1,13 @@
/*-------------------------------------------------------------------------
*
* execProcnode.c
- * contains dispatch functions which call the appropriate "initialize",
- * "get a tuple", and "cleanup" routines for the given node type.
- * If the node has children, then it will presumably call ExecInitNode,
- * ExecProcNode, or ExecEndNode on its subnodes and do the appropriate
- * processing.
+ * Contains dispatch functions ExecInitNode(), ExecProcNode(), and
+ * ExecEndNode(), which call the appropriate "initialize", "get a tuple",
+ * and "cleanup" routines, respectively, for the given node type.
+ *
+ * While the first two process the node's children recursively, ExecEndNode()
+ * is only concerned with the cleaning of the node itself while the children
+ * are processed by the caller.
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -49,7 +51,9 @@
* Eventually this calls ExecInitNode() on the right and left subplans
* and so forth until the entire plan is initialized. The result
* of ExecInitNode() is a plan state tree built with the same structure
- * as the underlying plan tree.
+ * as the underlying plan tree. (The plan state nodes are also added to
+ * a list in the same order in which they are created for the final
+ * cleanup processing.)
*
* * Then when ExecutorRun() is called, it calls ExecutePlan() which calls
* ExecProcNode() repeatedly on the top node of the plan state tree.
@@ -61,14 +65,10 @@
* form the tuples it returns.
*
* * Eventually ExecSeqScan() stops returning tuples and the nest
- * loop join ends. Lastly, ExecutorEnd() calls ExecEndNode() which
- * calls ExecEndNestLoop() which in turn calls ExecEndNode() on
- * its subplans which result in ExecEndSeqScan().
+ * loop join ends. Lastly, ExecutorEnd() calls ExecEndPlan(), which
+ * in turn calls ExecEndNode() on all the nodes that were initialized:
+ * the two Seq Scans and the Nest Loop in this case.
*
- * This should show how the executor works by having
- * ExecInitNode(), ExecProcNode() and ExecEndNode() dispatch
- * their work to the appropriate node support routines which may
- * in turn call these routines themselves on their subplans.
*/
#include "postgres.h"
@@ -136,6 +136,9 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * As a side-effect, all PlanState nodes that are created are appended to
+ * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -411,6 +414,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /* And remember for the cleanup processing in ExecEndPlan(). */
+ estate->es_planstate_nodes = lappend(estate->es_planstate_nodes,
+ result);
+
return result;
}
@@ -545,29 +552,18 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
- * the query plan has been fully executed.
+ * Unlike ExecInitNode(), this does not recurse into child nodes, because
+ * they are processed separately. So the ExecEnd* routine for any given
+ * node type is only responsible for cleaning up its own resources.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ /* We only ever get called on nodes that were actually initialized. */
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..b567165003 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -154,6 +154,8 @@ CreateExecutorState(void)
estate->es_exprcontexts = NIL;
+ estate->es_planstate_nodes = NIL;
+
estate->es_subplanstates = NIL;
estate->es_auxmodifytables = NIL;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..e9d9ab6bdd 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4367,8 +4366,7 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closely separately. */
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..9148d7d3b1 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -376,30 +376,12 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /* Nothing to do as the nodes in appendplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..147592f7e2 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -168,33 +168,12 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..d58ee4f4e1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,10 +667,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/*
* release bitmaps and buffers if any
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..736852a0ae 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -186,33 +186,12 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..e6616dd718 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -309,9 +309,7 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..f7a69f185b 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -248,7 +248,7 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..d357ff0c47 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -288,7 +288,7 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..2badcc7e60 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,15 +226,12 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..edd2324384 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,18 +413,12 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..8078d7f229 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -879,11 +879,7 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..52b146cfb8 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1101,10 +1101,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..a75099dd73 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,7 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..55de8d3d65 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -386,7 +386,7 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..ef04e9a8e7 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -251,10 +251,7 @@ ExecEndMaterial(MaterialState *node)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..61578d4b5c 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1100,10 +1100,7 @@ ExecEndMemoize(MemoizeState *node)
*/
ExecFreeExprContext(&node->ss.ps);
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8aa64944c9 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -310,30 +310,12 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /* Nothing to do as the nodes in mergeplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..7b530d9088 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1654,11 +1654,7 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..bdbaa4753b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4397,7 +4397,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* ----------------------------------------------------------------
* ExecEndModifyTable
*
- * Shuts down the plan.
+ * Releases ModifyTable resources.
*
* Returns nothing of interest.
* ----------------------------------------------------------------
@@ -4461,10 +4461,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
EvalPlanQualEnd(&node->mt_epqstate);
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..5cfb50a366 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -374,11 +374,7 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..4a388220ee 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -330,10 +330,7 @@ ExecEndProjectSet(ProjectSetState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..aee31c7139 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -281,11 +281,7 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..a100b144be 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -250,10 +250,7 @@ ExecEndResult(ResultState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..f7db9a3415 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -590,7 +590,7 @@ ExecEndSetOp(SetOpState *node)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..078d041c40 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -317,10 +317,7 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..bc55a82fc3 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -179,10 +179,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
+ /* subplan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..50babacdc8 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -173,7 +173,7 @@ ExecEndUnique(UniqueState *node)
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..648cdadc32 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2681,7 +2681,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2714,8 +2713,7 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
/* -----------------
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..233fb6b4f9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -671,6 +671,8 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_planstate_nodes; /* "flat" list of PlanState nodes */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
--
2.35.3
[application/octet-stream] v43-0005-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 4-v43-0005-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From b04831f37758fd86bf1e0fb41d8bf1001de4e8a0 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v43 5/5] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4a4b4b7690..ae9f45355f 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1653,12 +1653,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index ee12235b2f..5ae993e29c 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -839,6 +839,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 20c1bacae1..c519a6d5dc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v43-0003-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 5-v43-0003-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From b6e4c896027f832ff0e5762795355a89fd93afb2 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v43 3/5] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fe003ded50..72f2b0c04f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v43-0004-Delay-locking-of-child-tables-in-cached-plans-un.patch (122.0K, 6-v43-0004-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 90e9a2935a97ff085f521a34dfd86b9800542ab1 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v43 4/5] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, InitPlan().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Also,
ExecutorStart() now returns true or false to indicate whether or not
QueryDesc.planstate points to a successfully initialized PlanState
tree. Call sites that use GetCachedPlan() to get the plan trees to
pass to the executor should now be prepared to retry in the cases
where ExecutorStart() returns false.
Given this new behavior, PortalStart() now must always perform
ExecutorStart() to be able to drop and recreate cached plans if
needed, which is currently only done so for single-query portals.
For multi-query portals, the QueryDescs that are now created during
PortalStart() are remembered in a new List field of Portal called
'qdescs' and allocated in a new memory context 'queryContext'.
PortalRunMulti() now simply performs ExecutorRun() on the
QueryDescs found in 'qdescs'.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 10 +-
src/backend/commands/createas.c | 9 +-
src/backend/commands/explain.c | 144 +++++---
src/backend/commands/extension.c | 5 +-
src/backend/commands/matview.c | 9 +-
src/backend/commands/portalcmds.c | 24 +-
src/backend/commands/prepare.c | 32 +-
src/backend/executor/README | 39 ++
src/backend/executor/execMain.c | 96 ++++-
src/backend/executor/execParallel.c | 18 +-
src/backend/executor/execPartition.c | 14 +
src/backend/executor/execProcnode.c | 20 +-
src/backend/executor/execUtils.c | 63 +++-
src/backend/executor/functions.c | 5 +-
src/backend/executor/nodeAgg.c | 2 +
src/backend/executor/nodeAppend.c | 25 ++
src/backend/executor/nodeBitmapAnd.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 4 +
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 5 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 4 +
src/backend/executor/nodeGather.c | 3 +
src/backend/executor/nodeGatherMerge.c | 2 +
src/backend/executor/nodeGroup.c | 2 +
src/backend/executor/nodeHash.c | 2 +
src/backend/executor/nodeHashjoin.c | 4 +
src/backend/executor/nodeIncrementalSort.c | 2 +
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 2 +
src/backend/executor/nodeLockRows.c | 2 +
src/backend/executor/nodeMaterial.c | 2 +
src/backend/executor/nodeMemoize.c | 2 +
src/backend/executor/nodeMergeAppend.c | 25 ++
src/backend/executor/nodeMergejoin.c | 4 +
src/backend/executor/nodeModifyTable.c | 7 +
src/backend/executor/nodeNestloop.c | 4 +
src/backend/executor/nodeProjectSet.c | 2 +
src/backend/executor/nodeRecursiveunion.c | 4 +
src/backend/executor/nodeResult.c | 2 +
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 2 +
src/backend/executor/nodeSort.c | 2 +
src/backend/executor/nodeSubqueryscan.c | 2 +
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 2 +
src/backend/executor/nodeWindowAgg.c | 2 +
src/backend/executor/spi.c | 51 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++
src/backend/tcop/postgres.c | 19 +-
src/backend/tcop/pquery.c | 344 +++++++++---------
src/backend/utils/cache/lsyscache.c | 21 ++
src/backend/utils/cache/plancache.c | 149 +++-----
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 19 +-
src/include/nodes/execnodes.h | 2 +
src/include/storage/lmgr.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 +
src/include/utils/portal.h | 3 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++-
.../expected/cached-plan-replan.out | 156 ++++++++
.../specs/cached-plan-replan.spec | 61 ++++
73 files changed, 1251 insertions(+), 410 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 55b957d251..1160a7326a 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -963,13 +963,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -992,6 +994,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..2b0d0a8ebd 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -568,7 +569,12 @@ BeginCopyTo(ParseState *pstate,
*
* ExecutorStart computes a result tupdesc for us
*/
- ExecutorStart(cstate->queryDesc, 0);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = ExecutorStart(cstate->queryDesc, 0);
+ Assert(plan_valid);
+ }
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..bb359bb190 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,12 +325,17 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
/* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ Assert(plan_valid);
+ }
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..954d83fb0a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,89 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated as we're doing that.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +602,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +620,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4896,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 4cc994ca31..477c299112 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -795,13 +795,16 @@ execute_sql_string(const char *sql)
if (stmt->utilityStmt == NULL)
{
QueryDesc *qdesc;
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ plan_valid = ExecutorStart(qdesc, 0);
+ Assert(plan_valid);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..fb0b29384c 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,17 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
/* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = ExecutorStart(queryDesc, 0);
+ Assert(plan_valid);
+ }
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..fdce72c9a5 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -143,9 +143,14 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, params, 0, GetActiveSnapshot());
- Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(portal->strategy == PORTAL_ONE_SELECT);
+ Assert(plan_valid);
+ }
/*
* We're done; the query won't actually be run until PerformPortalFetch is
@@ -249,6 +254,17 @@ PerformPortalClose(const char *name)
PortalDrop(portal, false);
}
+/*
+ * Release a portal's QueryDesc.
+ */
+void
+PortalQueryFinish(QueryDesc *queryDesc)
+{
+ ExecutorFinish(queryDesc);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+}
+
/*
* PortalCleanup
*
@@ -295,9 +311,7 @@ PortalCleanup(Portal portal)
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
- FreeQueryDesc(queryDesc);
+ PortalQueryFinish(queryDesc);
CurrentResourceOwner = saveResourceOwner;
}
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..07f0421182 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,16 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if portal->plan_valid is false which tells that the
+ * cached plan was found to have been invalidated when initializing one of
+ * the plan trees contained in it.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +582,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +626,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +648,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 67a5c1769b..f0312376c5 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,39 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to executor-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree (after executor-initialization-pruning, if any,
+has been performed), which it does by looking at CachedPlan.is_valid of the
+CachedPlan passed to it. If the plan tree is indeed stale (is_valid=false),
+the executor must give up continuing to initialize it any further and return
+to the caller letting it know that the execution must be retried with a new
+plan tree.
+
Query Processing Control Flow
-----------------------------
@@ -316,6 +349,12 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 235bb52ccc..4a4b4b7690 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(EState *estate);
@@ -128,7 +128,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +140,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +264,11 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -620,6 +623,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,9 +843,26 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ *
+ * Normally, the plan tree given in queryDesc->plannedstmt is known to be
+ * valid in a race-free manner, that is, all relations contained in
+ * plannedstmt->relationOids would have already been locked. That is not the
+ * case however if the plannedstmt comes from a CachedPlan, one given in
+ * queryDesc->cplan. That's because GetCachedPlan() only locks the tables
+ * that are mentioned in the original query but not the child tables, which
+ * would have been added to the plan by the planner. In that case, locks on
+ * child tables will be taken when initializing their Scan nodes in
+ * ExecInitNode() to be done here. If the CachedPlan gets invalidated as
+ * those locks are taken, plan tree initialization is suspended at the point
+ * where the invalidation is first detected, queryDesc->planstate will be set
+ * to NULL, and queryDesc->plan_valid to false. Callers must retry the
+ * execution after creating a new CachedPlan in that case, after properly
+ * releasing the resources of this QueryDesc, which includes calling
+ * ExecutorFinish() and ExecutorEnd() on the EState contained therein.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,7 +870,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +881,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +918,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +987,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +1005,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1052,18 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ return true;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_inited_plannodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ return false;
}
/*
@@ -1426,7 +1481,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2856,7 +2911,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2943,6 +2999,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2986,6 +3048,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3008,6 +3076,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..d32bc74609 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1430,7 +1439,12 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = ExecutorStart(queryDesc, fpes->eflags);
+ Assert(plan_valid);
+ }
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 653f74cf58..2dcacafd03 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,10 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
*
- * As a side-effect, all PlanState nodes that are created are appended to
- * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * As a side-effect, all PlanState nodes that are successfully created are
+ * appended to estate->es_planstate_nodes for the cleanup processing in
+ * ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -391,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index b567165003..ee12235b2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -806,7 +806,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -819,15 +837,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -835,6 +844,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -850,6 +891,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..bc09ef992c 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -857,12 +858,14 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
* lazyEval mode for any statement that could possibly queue triggers.
*/
int eflags;
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
if (es->lazyEval)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+ plan_valid = ExecutorStart(es->qd, eflags);
+ Assert(plan_valid);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index e9d9ab6bdd..9553a85115 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 9148d7d3b1..b0cae25b33 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,27 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +168,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +244,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 147592f7e2..53afcef21c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d58ee4f4e1..388a02ec99 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -760,11 +760,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 736852a0ae..425f22ee48 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index e6616dd718..71495313db 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index f7a69f185b..c5652aeb2d 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index d357ff0c47..1191b9e420 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 2badcc7e60..b4c3044c1f 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index edd2324384..b2119febb6 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 8078d7f229..d5ff80660e 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 52b146cfb8..785896e5ea 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a75099dd73..a1fc36a3f0 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 55de8d3d65..ff86a82b92 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index ef04e9a8e7..8d02ac0ccb 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 61578d4b5c..a994d48fb2 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 8aa64944c9..18808c19ae 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,27 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Must take locks on child tables if running a cached plan, because
+ * GetCachedPlan() would've only locked the root parent named in the
+ * query.
+ *
+ * First lock non-leaf partitions before doing pruning if any. Even when
+ * no pruning is to be done, non-leaf partitions still must be locked
+ * explicitly like this, because they're not referenced elsewhere in
+ * the plan tree. XXX - OTOH, non-leaf partitions mentioned in
+ * part_prune_info, if any, would be opened by ExecInitPartitionPruning()
+ * using ExecGetRangeTableRelation() which locks child tables, redundantly
+ * in this case.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +116,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +174,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 7b530d9088..0d92ec278a 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index bdbaa4753b..2245a67397 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 5cfb50a366..e24554f4f8 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index 4a388220ee..863bf2cc65 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index aee31c7139..3f3de771d0 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index a100b144be..f2206af451 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index f7db9a3415..3535aa298c 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index 078d041c40..547203ebfd 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index bc55a82fc3..8b6629c939 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 50babacdc8..ae9af8f21e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 648cdadc32..77de2d0c22 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..cfee208719 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,24 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2563,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2673,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2668,14 +2681,31 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2880,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2926,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..cf27fa3968 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,16 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if portal->plan_valid is false which tells that the cached
+ * plan was found to have been invalidated when initializing one of the
+ * plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..528f795d4f 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -426,19 +343,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +367,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +389,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -493,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -501,30 +421,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated as we're doing that.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +499,81 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc object. DestReceiver will
+ * be set in PortalRunMulti().
+ */
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated as
+ * we're doing that.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ Assert(queryDesc->cplan);
+ PortalQueryFinish(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,19 +586,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1193,7 +1186,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1207,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1227,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1255,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1342,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index d67cd9a405..84a354a701 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,14 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * Note though that if the plan contains any child relations that would have
+ * been added by the planner, which would not have been locked yet (because
+ * AcquirePlannerLocks() only locks relations that would be present in the
+ * range table before entering the planner), the plan could go stale before
+ * it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables, and if it is not,
+ * ask the caller to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +811,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1130,15 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, that is, only the locks on the tables mentioned in the query have
+ * been taken. If any of those tables have inheritance/partition tables, the
+ * executor must also lock them before executing the plan and if the plan gets
+ * invalidated as a result of taking those locks, must ask the caller to get
+ * a new plan by calling here again. Locking of the child tables must be
+ * deferred to the executor like this, because not all child tables may need
+ * to be locked; some may get pruned during the executor plan initialization
+ * phase (InitPlan()).
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1362,8 +1371,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1739,58 +1748,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..edf2f13d04 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -72,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -197,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 233fb6b4f9..20c1bacae1 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..c83a67fea3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..e7e2fb0c3f 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
@@ -242,6 +244,7 @@ extern void PortalDefineQuery(Portal portal,
CommandTag commandTag,
List *stmts,
CachedPlan *cplan);
+extern void PortalQueryFinish(QueryDesc *queryDesc);
extern PlannedStmt *PortalGetPrimaryStmt(Portal portal);
extern void PortalCreateHoldStore(Portal portal);
extern void PortalHashTableDeleteAll(void);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-03 08:37 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-08-03 08:37 UTC (permalink / raw)
To: Thom Brown <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>
On Wed, Aug 2, 2023 at 10:39 PM Amit Langote <[email protected]> wrote:
> Having extracted the ExecEndNode() change, I'm also starting to feel
> inclined to extract a couple of other bits from the main patch as
> separate patches, such as moving the ExecutorStart() call from
> PortalRun() to PortalStart() for the multi-query portals. I'll do
> that in the next version.
Here's a patch set where the refactoring to move the ExecutorStart()
calls to be closer to GetCachedPlan() (for the call sites that use a
CachedPlan) is extracted into a separate patch, 0002. Its commit
message notes an aspect of this refactoring that I feel a bit nervous
about -- needing to also move the CommandCounterIncrement() call from
the loop in PortalRunMulti() to PortalStart() which now does
ExecutorStart() for the PORTAL_MULTI_QUERY case.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v44-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 2-v44-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From 26df10ea36b2089d59129b066d3dfaedb3aa5e0c Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v44 4/6] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fe003ded50..72f2b0c04f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v44-0006-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 3-v44-0006-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 4f73533573cb5959b15268455605816c0316d0e6 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v44 6/6] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index dcfbf58495..c574cd3cdc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1646,12 +1646,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index ee12235b2f..5ae993e29c 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -839,6 +839,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 20c1bacae1..c519a6d5dc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v44-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch (25.8K, 4-v44-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch)
download | inline diff:
From 0d1067505ed1c49d4a75ad5d7f4eec4a19d7b5d6 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 3 Aug 2023 12:34:31 +0900
Subject: [PATCH v44 2/6] Refactoring to move ExecutorStart() calls to be near
GetCachedPlan()
An upcoming patch will make ExecutorStart() detect the invalidation
of a CachedPlan when initializing the plan tree contained in it. A
caller must retry with a new CachedPlan when ExecutorStart() detects
an invalidation. Having the ExecutorStart() in the same or nearby
as GetCachedPlan() makes it more convenient to implement the replan
loop.
The following sites have thus been modified:
* The ExecutorStart() call in ExplainOnePlan() is moved, along with
CreateQueryDesc(), into a new function ExplainQueryDesc(), which its
callers now call before calling it.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and the associated
memory context field. One unintended consequence is that the
CommandCounterIncrement() between queries in PORTAL_MULTI_QUERY
cases is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still seems to work because the Snapshot
registered in QueryDesc/EState is updated to account for the
CCI().
---
src/backend/commands/explain.c | 121 ++++++-----
src/backend/commands/prepare.c | 12 +-
src/backend/executor/spi.c | 27 +--
src/backend/tcop/pquery.c | 311 +++++++++++++----------------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 6 +-
src/include/utils/portal.h | 2 +
7 files changed, 250 insertions(+), 238 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..59d57f9c10 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,77 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /* Call ExecutorStart to prepare the plan for execution. */
+ ExecutorStart(queryDesc, eflags);
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +590,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
- Assert(plannedstmt->commandType != CMD_UTILITY);
-
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +608,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..1e9a98ad6e 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -639,8 +639,16 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, queryString,
+ into, es, paramLI, queryEnv);
+ Assert(queryDesc != NULL);
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..d36ca35d3a 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -2661,6 +2661,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2674,8 +2675,17 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
+
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2860,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2906,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..701808f303 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -116,86 +111,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -435,10 +350,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +362,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +384,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -489,8 +403,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
portal->sourceText,
@@ -501,30 +415,41 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
+ /* Call ExecutorStart to prepare the plan for execution. */
ExecutorStart(queryDesc, myeflags);
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +461,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +483,69 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,7 +558,6 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
@@ -604,7 +567,6 @@ PortalStart(Portal portal, ParamListInfo params,
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
portal->status = PORTAL_READY;
}
@@ -1193,7 +1155,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1176,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1196,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1224,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1311,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..08ea852b65 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v44-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 5-v44-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 10c7bbe9f1a489d0fcfeaf027a7df919fed490c8 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v44 3/6] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..8ac1d3909b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v44-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch (101.6K, 6-v44-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From a51b753af1c38e1a7750e9f09738d304ebd7de07 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v44 5/6] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Also,
ExecutorStart() now returns true or false to indicate whether or not
QueryDesc.planstate points to a successfully initialized PlanState
tree. Call sites that use GetCachedPlan() to get the plan trees to
pass to the executor should now be prepared to retry in the cases
where ExecutorStart() returns false.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 7 +-
src/backend/commands/createas.c | 10 +-
src/backend/commands/explain.c | 33 +++-
src/backend/commands/extension.c | 4 +-
src/backend/commands/matview.c | 10 +-
src/backend/commands/portalcmds.c | 5 +-
src/backend/commands/prepare.c | 23 ++-
src/backend/executor/README | 37 +++++
src/backend/executor/execMain.c | 89 ++++++++--
src/backend/executor/execParallel.c | 14 +-
src/backend/executor/execPartition.c | 14 ++
src/backend/executor/execProcnode.c | 20 ++-
src/backend/executor/execUtils.c | 63 +++++--
src/backend/executor/functions.c | 5 +-
src/backend/executor/nodeAgg.c | 2 +
src/backend/executor/nodeAppend.c | 23 +++
src/backend/executor/nodeBitmapAnd.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 4 +
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 5 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 4 +
src/backend/executor/nodeGather.c | 3 +
src/backend/executor/nodeGatherMerge.c | 2 +
src/backend/executor/nodeGroup.c | 2 +
src/backend/executor/nodeHash.c | 2 +
src/backend/executor/nodeHashjoin.c | 4 +
src/backend/executor/nodeIncrementalSort.c | 2 +
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 2 +
src/backend/executor/nodeLockRows.c | 2 +
src/backend/executor/nodeMaterial.c | 2 +
src/backend/executor/nodeMemoize.c | 2 +
src/backend/executor/nodeMergeAppend.c | 23 +++
src/backend/executor/nodeMergejoin.c | 4 +
src/backend/executor/nodeModifyTable.c | 7 +
src/backend/executor/nodeNestloop.c | 4 +
src/backend/executor/nodeProjectSet.c | 2 +
src/backend/executor/nodeRecursiveunion.c | 4 +
src/backend/executor/nodeResult.c | 2 +
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 2 +
src/backend/executor/nodeSort.c | 2 +
src/backend/executor/nodeSubqueryscan.c | 2 +
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 2 +
src/backend/executor/nodeWindowAgg.c | 2 +
src/backend/executor/spi.c | 26 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++++
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 49 +++++-
src/backend/utils/cache/lsyscache.c | 21 +++
src/backend/utils/cache/plancache.c | 156 +++++++-----------
src/include/commands/explain.h | 3 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 19 ++-
src/include/nodes/execnodes.h | 2 +
src/include/storage/lmgr.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 ++
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 156 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
71 files changed, 984 insertions(+), 189 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 55b957d251..1160a7326a 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -963,13 +963,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -992,6 +994,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..916d6dced3 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -567,8 +568,10 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..e5cce4c07c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,12 +325,16 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 59d57f9c10..6171a20fe2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -416,7 +416,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
- queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
params, queryEnv);
Assert(queryDesc);
@@ -429,9 +429,11 @@ ExplainOneQuery(Query *query, int cursorOptions,
/*
* ExplainQueryDesc
* Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
*/
QueryDesc *
-ExplainQueryDesc(PlannedStmt *stmt,
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv)
{
@@ -467,7 +469,7 @@ ExplainQueryDesc(PlannedStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(stmt, queryString,
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
@@ -481,8 +483,18 @@ ExplainQueryDesc(PlannedStmt *stmt,
if (into)
eflags |= GetIntoRelEFlags(into);
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, eflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
return queryDesc;
}
@@ -4884,6 +4896,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 4cc994ca31..8a0859a355 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..38795ce7ca 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,16 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..5120f93414 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,10 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 1e9a98ad6e..156c3c5fee 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if the cached plan was found to have been invalidated
+ * when initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -642,9 +650,14 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
QueryDesc *queryDesc;
- queryDesc = ExplainQueryDesc(pstmt, queryString,
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
into, es, paramLI, queryEnv);
- Assert(queryDesc != NULL);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
queryEnv, &planduration,
(es->buffers ? &bufusage : NULL));
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 67a5c1769b..5113523bb9 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,37 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to execution-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree after execution-initialization-pruning has been
+performed. It does that by looking at CachedPlan.is_valid of the CachedPlan
+passed to it. If the plan tree is indeed stale (is_valid=false), the executor
+must give up continuing to initialize it any further and return to the caller
+letting it know that the execution must be retried with a new plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,6 +347,12 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 235bb52ccc..dcfbf58495 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(EState *estate);
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,11 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -620,6 +630,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,9 +850,12 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,7 +863,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +874,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +911,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +980,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +998,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1045,18 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ return true;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_planstate_nodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ return false;
}
/*
@@ -1426,7 +1474,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2856,7 +2904,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2943,6 +2992,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2986,6 +3041,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3008,6 +3069,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..bfa2a8ec18 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1430,7 +1439,8 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 653f74cf58..2dcacafd03 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,10 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
*
- * As a side-effect, all PlanState nodes that are created are appended to
- * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * As a side-effect, all PlanState nodes that are successfully created are
+ * appended to estate->es_planstate_nodes for the cleanup processing in
+ * ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -391,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index b567165003..ee12235b2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -806,7 +806,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -819,15 +837,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -835,6 +844,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -850,6 +891,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..4ddf4fd7a9 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -862,7 +863,9 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index e9d9ab6bdd..9553a85115 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 9148d7d3b1..222434a84d 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,25 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +166,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +242,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 147592f7e2..53afcef21c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d58ee4f4e1..388a02ec99 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -760,11 +760,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 736852a0ae..425f22ee48 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index e6616dd718..71495313db 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index f7a69f185b..c5652aeb2d 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index d357ff0c47..1191b9e420 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 2badcc7e60..b4c3044c1f 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index edd2324384..b2119febb6 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 8078d7f229..d5ff80660e 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 52b146cfb8..785896e5ea 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a75099dd73..a1fc36a3f0 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 55de8d3d65..ff86a82b92 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index ef04e9a8e7..8d02ac0ccb 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 61578d4b5c..a994d48fb2 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 8aa64944c9..14d07c30e8 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,25 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +114,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +172,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 7b530d9088..0d92ec278a 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index bdbaa4753b..2245a67397 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 5cfb50a366..e24554f4f8 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index 4a388220ee..863bf2cc65 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index aee31c7139..3f3de771d0 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index a100b144be..f2206af451 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index f7db9a3415..3535aa298c 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index 078d041c40..547203ebfd 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index bc55a82fc3..8b6629c939 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 50babacdc8..ae9af8f21e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 648cdadc32..77de2d0c22 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index d36ca35d3a..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2669,6 +2680,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
@@ -2682,10 +2694,16 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
else
eflags = EXEC_FLAG_SKIP_TRIGGERS;
- ExecutorStart(qdesc, eflags);
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
-
FreeQueryDesc(qdesc);
}
else
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..88724a8d67 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 701808f303..48cd6f4304 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -60,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -72,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -341,10 +343,12 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
@@ -353,6 +357,7 @@ PortalStart(Portal portal, ParamListInfo params,
MemoryContext oldContext;
QueryDesc *queryDesc;
int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -407,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -431,8 +437,19 @@ PortalStart(Portal portal, ParamListInfo params,
else
myeflags = eflags;
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
* This tells PortalCleanup to shut down the executor, though
@@ -525,7 +542,7 @@ PortalStart(Portal portal, ParamListInfo params,
* Create the QueryDesc. DestReceiver will be set in
* PortalRunMulti() before calling ExecutorRun().
*/
- queryDesc = CreateQueryDesc(plan,
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -541,7 +558,20 @@ PortalStart(Portal portal, ParamListInfo params,
if (is_utility)
continue;
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
PopActiveSnapshot();
}
}
@@ -563,12 +593,15 @@ PortalStart(Portal portal, ParamListInfo params,
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index d67cd9a405..c5a7616b33 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,15 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan contains any child relations that would have been added by the
+ * planner, they would not have been locked yet, because AcquirePlannerLocks()
+ * only locks relations that would be present in the original query's range
+ * table (that is, before entering the planner). So, the plan could go stale
+ * before it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables during the plan
+ * initialization phase, and if it is not, ask the caller to recreate the
+ * plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, because they will not have been locked as here we only lock the
+ * tables mentioned in the original query. Inheritance/partition child tables
+ * are locked by the executor when initializing the plan tree and if the plan
+ * gets invalidated as a result of taking those locks, the executor must ask
+ * the caller to get a new plan by calling here again. Locking of the child
+ * tables is deferred to the executor in this manner, because not all child
+ * tables may need to be locked as some may get pruned during the executor
+ * plan initialization which performs initial pruing on any nodes that
+ * support partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1164,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1362,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1739,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 08ea852b65..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,7 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv);
extern void ExplainOnePlan(QueryDesc *queryDesc,
@@ -108,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..edf2f13d04 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -72,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -197,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 233fb6b4f9..20c1bacae1 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..c83a67fea3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v44-0001-Make-PlanState-tree-cleanup-non-recursive.patch (28.2K, 7-v44-0001-Make-PlanState-tree-cleanup-non-recursive.patch)
download | inline diff:
From a55bd363690bc4c28047e4b874ce80384e37c49d Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 1 Aug 2023 11:36:24 +0900
Subject: [PATCH v44 1/6] Make PlanState tree cleanup non-recursive
With this change, node type specific subroutines of ExecEndNode()
are no longer required to also clean up the child nodes of a given
node, only its own stuff. Instead, ExecEndPlan() calls
ExecInitNode() directly for each node in the PlanState tree by
iterating over a list (EState.es_planstate_nodes) of all those nodes
built during the ExecInitNode() traversal of the tree.
This changes the order in which the nodes get cleaned up, because
they are now cleaned up in the order in which they are added into
the list which is from leaf-level up to the root, whereas with the
current recursive approach cleanup occurs from the root to the
leaves. The change seems harmless though, because there isn't
necessarily any coupling between of the cleanup actions of parent
and child nodes.
The main motivation behind this change is to allow the cases in
the future where ExecInitNode() traversal of the plan tree may
be aborted in the middle resulting in a partially initialized
PlanState tree. Dealing with that case by making the cleanup
phase walk over a list of successfully initialized nodes seems
better / more robust than making the individual ExecEndNode()
subroutines deal with partially valid PlanState nodes.
---
src/backend/executor/README | 4 +-
src/backend/executor/execMain.c | 36 +++++++-------
src/backend/executor/execProcnode.c | 56 ++++++++++------------
src/backend/executor/execUtils.c | 2 +
src/backend/executor/nodeAgg.c | 4 +-
src/backend/executor/nodeAppend.c | 20 +-------
src/backend/executor/nodeBitmapAnd.c | 23 +--------
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeBitmapOr.c | 23 +--------
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeGather.c | 2 +-
src/backend/executor/nodeGatherMerge.c | 2 +-
src/backend/executor/nodeGroup.c | 5 +-
src/backend/executor/nodeHash.c | 8 +---
src/backend/executor/nodeHashjoin.c | 6 +--
src/backend/executor/nodeIncrementalSort.c | 5 +-
src/backend/executor/nodeLimit.c | 2 +-
src/backend/executor/nodeLockRows.c | 2 +-
src/backend/executor/nodeMaterial.c | 5 +-
src/backend/executor/nodeMemoize.c | 5 +-
src/backend/executor/nodeMergeAppend.c | 20 +-------
src/backend/executor/nodeMergejoin.c | 6 +--
src/backend/executor/nodeModifyTable.c | 7 +--
src/backend/executor/nodeNestloop.c | 6 +--
src/backend/executor/nodeProjectSet.c | 5 +-
src/backend/executor/nodeRecursiveunion.c | 6 +--
src/backend/executor/nodeResult.c | 5 +-
src/backend/executor/nodeSetOp.c | 2 +-
src/backend/executor/nodeSort.c | 5 +-
src/backend/executor/nodeSubqueryscan.c | 5 +-
src/backend/executor/nodeUnique.c | 2 +-
src/backend/executor/nodeWindowAgg.c | 4 +-
src/include/nodes/execnodes.h | 2 +
33 files changed, 80 insertions(+), 214 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..67a5c1769b 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -310,13 +310,13 @@ This is a sketch of control flow for full query processing:
AfterTriggerEndQuery
ExecutorEnd
- ExecEndNode --- recursively releases resources
+ ExecEndPlan --- releases plan resources
FreeExecutorState
frees per-query context and child contexts
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..235bb52ccc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -82,7 +82,7 @@ ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
-static void ExecEndPlan(PlanState *planstate, EState *estate);
+static void ExecEndPlan(EState *estate);
static void ExecutePlan(EState *estate, PlanState *planstate,
bool use_parallel_mode,
CmdType operation,
@@ -500,7 +500,7 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
*/
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndPlan(queryDesc->planstate, estate);
+ ExecEndPlan(estate);
/* do away with our snapshots */
UnregisterSnapshot(estate->es_snapshot);
@@ -1499,23 +1499,21 @@ ExecPostprocessPlan(EState *estate)
* ----------------------------------------------------------------
*/
static void
-ExecEndPlan(PlanState *planstate, EState *estate)
+ExecEndPlan(EState *estate)
{
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (es_plannedstmt->planTree) and those in subplans
+ * (es_plannedstmt->subplans).
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -3030,13 +3028,17 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (epqstate->plan) and those in subplans
+ * (es_plannedstmt->subplans).
+ */
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..653f74cf58 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -1,11 +1,13 @@
/*-------------------------------------------------------------------------
*
* execProcnode.c
- * contains dispatch functions which call the appropriate "initialize",
- * "get a tuple", and "cleanup" routines for the given node type.
- * If the node has children, then it will presumably call ExecInitNode,
- * ExecProcNode, or ExecEndNode on its subnodes and do the appropriate
- * processing.
+ * Contains dispatch functions ExecInitNode(), ExecProcNode(), and
+ * ExecEndNode(), which call the appropriate "initialize", "get a tuple",
+ * and "cleanup" routines, respectively, for the given node type.
+ *
+ * While the first two process the node's children recursively, ExecEndNode()
+ * is only concerned with the cleaning of the node itself while the children
+ * are processed by the caller.
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -49,7 +51,9 @@
* Eventually this calls ExecInitNode() on the right and left subplans
* and so forth until the entire plan is initialized. The result
* of ExecInitNode() is a plan state tree built with the same structure
- * as the underlying plan tree.
+ * as the underlying plan tree. (The plan state nodes are also added to
+ * a list in the same order in which they are created for the final
+ * cleanup processing.)
*
* * Then when ExecutorRun() is called, it calls ExecutePlan() which calls
* ExecProcNode() repeatedly on the top node of the plan state tree.
@@ -61,14 +65,10 @@
* form the tuples it returns.
*
* * Eventually ExecSeqScan() stops returning tuples and the nest
- * loop join ends. Lastly, ExecutorEnd() calls ExecEndNode() which
- * calls ExecEndNestLoop() which in turn calls ExecEndNode() on
- * its subplans which result in ExecEndSeqScan().
+ * loop join ends. Lastly, ExecutorEnd() calls ExecEndPlan(), which
+ * in turn calls ExecEndNode() on all the nodes that were initialized:
+ * the two Seq Scans and the Nest Loop in this case.
*
- * This should show how the executor works by having
- * ExecInitNode(), ExecProcNode() and ExecEndNode() dispatch
- * their work to the appropriate node support routines which may
- * in turn call these routines themselves on their subplans.
*/
#include "postgres.h"
@@ -136,6 +136,9 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * As a side-effect, all PlanState nodes that are created are appended to
+ * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -411,6 +414,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /* And remember for the cleanup processing in ExecEndPlan(). */
+ estate->es_planstate_nodes = lappend(estate->es_planstate_nodes,
+ result);
+
return result;
}
@@ -545,29 +552,18 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
- * the query plan has been fully executed.
+ * Unlike ExecInitNode(), this does not recurse into child nodes, because
+ * they are processed separately. So the ExecEnd* routine for any given
+ * node type is only responsible for cleaning up its own resources.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ /* We only ever get called on nodes that were actually initialized. */
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..b567165003 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -154,6 +154,8 @@ CreateExecutorState(void)
estate->es_exprcontexts = NIL;
+ estate->es_planstate_nodes = NIL;
+
estate->es_subplanstates = NIL;
estate->es_auxmodifytables = NIL;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..e9d9ab6bdd 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4367,8 +4366,7 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closely separately. */
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..9148d7d3b1 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -376,30 +376,12 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /* Nothing to do as the nodes in appendplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..147592f7e2 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -168,33 +168,12 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..d58ee4f4e1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,10 +667,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/*
* release bitmaps and buffers if any
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..736852a0ae 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -186,33 +186,12 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..e6616dd718 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -309,9 +309,7 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..f7a69f185b 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -248,7 +248,7 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..d357ff0c47 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -288,7 +288,7 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..2badcc7e60 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,15 +226,12 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..edd2324384 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,18 +413,12 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..8078d7f229 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -879,11 +879,7 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..52b146cfb8 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1101,10 +1101,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..a75099dd73 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,7 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..55de8d3d65 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -386,7 +386,7 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..ef04e9a8e7 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -251,10 +251,7 @@ ExecEndMaterial(MaterialState *node)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..61578d4b5c 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1100,10 +1100,7 @@ ExecEndMemoize(MemoizeState *node)
*/
ExecFreeExprContext(&node->ss.ps);
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8aa64944c9 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -310,30 +310,12 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /* Nothing to do as the nodes in mergeplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..7b530d9088 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1654,11 +1654,7 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..bdbaa4753b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4397,7 +4397,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* ----------------------------------------------------------------
* ExecEndModifyTable
*
- * Shuts down the plan.
+ * Releases ModifyTable resources.
*
* Returns nothing of interest.
* ----------------------------------------------------------------
@@ -4461,10 +4461,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
EvalPlanQualEnd(&node->mt_epqstate);
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..5cfb50a366 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -374,11 +374,7 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..4a388220ee 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -330,10 +330,7 @@ ExecEndProjectSet(ProjectSetState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..aee31c7139 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -281,11 +281,7 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..a100b144be 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -250,10 +250,7 @@ ExecEndResult(ResultState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..f7db9a3415 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -590,7 +590,7 @@ ExecEndSetOp(SetOpState *node)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..078d041c40 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -317,10 +317,7 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..bc55a82fc3 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -179,10 +179,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
+ /* subplan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..50babacdc8 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -173,7 +173,7 @@ ExecEndUnique(UniqueState *node)
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..648cdadc32 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2681,7 +2681,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2714,8 +2713,7 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
/* -----------------
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..233fb6b4f9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -671,6 +671,8 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_planstate_nodes; /* "flat" list of PlanState nodes */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-07 15:36 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 2 replies; 31+ messages in thread
From: Robert Haas @ 2023-08-07 15:36 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Thom Brown <[email protected]>; Daniel Gustafsson <[email protected]>; Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers
On Thu, Aug 3, 2023 at 4:37 AM Amit Langote <[email protected]> wrote:
> Here's a patch set where the refactoring to move the ExecutorStart()
> calls to be closer to GetCachedPlan() (for the call sites that use a
> CachedPlan) is extracted into a separate patch, 0002. Its commit
> message notes an aspect of this refactoring that I feel a bit nervous
> about -- needing to also move the CommandCounterIncrement() call from
> the loop in PortalRunMulti() to PortalStart() which now does
> ExecutorStart() for the PORTAL_MULTI_QUERY case.
I spent some time today reviewing 0001. Here are a few thoughts and
notes about things that I looked at.
First, I wondered whether it was really adequate for ExecEndPlan() to
just loop over estate->es_plan_nodes and call it good. Put
differently, is it possible that we could ever have more than one
relevant EState, say for a subplan or an EPQ execution or something,
so that this loop wouldn't cover everything? I found nothing to make
me think that this is a real danger.
Second, I wondered whether the ordering of cleanup operations could be
an issue. Right now, a node can position cleanup code before, after,
or both before and after recursing to child nodes, whereas with this
design change, the cleanup code will always be run before recursing to
child nodes. Here, I think we have problems. Both ExecGather and
ExecEndGatherMerge intentionally clean up the children before the
parent, so that the child shutdown happens before
ExecParallelCleanup(). Based on the comment and commit
acf555bc53acb589b5a2827e65d655fa8c9adee0, this appears to be
intentional, and you can sort of see why from looking at the stuff
that happens in ExecParallelCleanup(). If the instrumentation data
vanishes before the child nodes have a chance to clean things up,
maybe EXPLAIN ANALYZE won't reflect that instrumentation any more. If
the DSA vanishes, maybe we'll crash if we try to access it. If we
actually reach DestroyParallelContext(), we're just going to start
killing the workers. None of that sounds like what we want.
The good news, of a sort, is that I think this might be the only case
of this sort of problem. Most nodes recurse at the end, after doing
all the cleanup, so the behavior won't change. Moreover, even if it
did, most cleanup operations look pretty localized -- they affect only
the node itself, and not its children. A somewhat interesting case is
nodes associated with subplans. Right now, because of the coding of
ExecEndPlan, nodes associated with subplans are all cleaned up at the
very end, after everything that's not inside of a subplan. But with
this change, they'd get cleaned up in the order of initialization,
which actually seems more natural, as long as it doesn't break
anything, which I think it probably won't, since as I mention in most
cases node cleanup looks quite localized, i.e. it doesn't care whether
it happens before or after the cleanup of other nodes.
I think something will have to be done about the parallel query stuff,
though. I'm not sure exactly what. It is a little weird that Gather
and Gather Merge treat starting and killing workers as a purely
"private matter" that they can decide to handle without the executor
overall being very much aware of it. So maybe there's a way that some
of the cleanup logic here could be hoisted up into the general
executor machinery, that is, first end all the nodes, and then go
back, and end all the parallelism using, maybe, another list inside of
the estate. However, I think that the existence of ExecShutdownNode()
is a complication here -- we need to make sure that we don't break
either the case where that happen before overall plan shutdown, or the
case where it doesn't.
Third, a couple of minor comments on details of how you actually made
these changes in the patch set. Personally, I would remove all of the
"is closed separately" comments that you added. I think it's a
violation of the general coding principle that you should make the
code look like it's always been that way. Sure, in the immediate
future, people might wonder why you don't need to recurse, but 5 or 10
years from now that's just going to be clutter. Second, in the cases
where the ExecEndNode functions end up completely empty, I would
suggest just removing the functions entirely and making the switch
that dispatches on the node type have a switch case that lists all the
nodes that don't need a callback here and say /* Nothing do for these
node types */ break;. This will save a few CPU cycles and I think it
will be easier to read as well.
Fourth, I wonder whether we really need this patch at all. I initially
thought we did, because if we abandon the initialization of a plan
partway through, then we end up with a plan that is in a state that
previously would never have occurred, and we still have to be able to
clean it up. However, perhaps it's a difference without a distinction.
Say we have a partial plan tree, where not all of the PlanState nodes
ever got created. We then just call the existing version of
ExecEndPlan() on it, with no changes. What goes wrong? Sure, we might
call ExecEndNode() on some null pointers where in the current world
there would always be valid pointers, but ExecEndNode() will handle
that just fine, by doing nothing for those nodes, because it starts
with a NULL-check.
Another alternative design might be to switch ExecEndNode to use
planstate_tree_walker to walk the node tree, removing the walk from
the node-type-specific functions as in this patch, and deleting the
end-node functions that are no longer required altogether, as proposed
above. I somehow feel that this would be cleaner than the status quo,
but here again, I'm not sure we really need it. planstate_tree_walker
would just pass over any NULL pointers that it found without doing
anything, but the current code does that too, so while this might be
more beautiful than what we have now, I'm not sure that there's any
real reason to do it. The fact that, like the current patch, it would
change the order in which nodes are cleaned up is also an issue -- the
Gather/Gather Merge ordering issues might be easier to handle this way
with some hack in ExecEndNode() than they are with the design you have
now, but we'd still have to do something about them, I believe.
Sorry if this is a bit of a meandering review, but those are my thoughts.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-07 15:44 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: Tom Lane @ 2023-08-07 15:44 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Amit Langote <[email protected]>; Thom Brown <[email protected]>; Daniel Gustafsson <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> Second, I wondered whether the ordering of cleanup operations could be
> an issue. Right now, a node can position cleanup code before, after,
> or both before and after recursing to child nodes, whereas with this
> design change, the cleanup code will always be run before recursing to
> child nodes. Here, I think we have problems. Both ExecGather and
> ExecEndGatherMerge intentionally clean up the children before the
> parent, so that the child shutdown happens before
> ExecParallelCleanup(). Based on the comment and commit
> acf555bc53acb589b5a2827e65d655fa8c9adee0, this appears to be
> intentional, and you can sort of see why from looking at the stuff
> that happens in ExecParallelCleanup().
Right, I doubt that changing that is going to work out well.
Hash joins might have issues with it too.
Could it work to make the patch force child cleanup before parent,
instead of after? Or would that break other places?
On the whole though I think it's probably a good idea to leave
parent nodes in control of the timing, so I kind of side with
your later comment about whether we want to change this at all.
regards, tom lane
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-07 16:25 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Robert Haas @ 2023-08-07 16:25 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Amit Langote <[email protected]>; Thom Brown <[email protected]>; Daniel Gustafsson <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers
On Mon, Aug 7, 2023 at 11:44 AM Tom Lane <[email protected]> wrote:
> Right, I doubt that changing that is going to work out well.
> Hash joins might have issues with it too.
I thought about the case, because Hash and Hash Join are such closely
intertwined nodes, but I don't see any problem there. It doesn't
really look like it would matter in what order things got cleaned up.
Unless I'm missing something, all of the data structures are just
independent things that we have to get rid of sometime.
> Could it work to make the patch force child cleanup before parent,
> instead of after? Or would that break other places?
To me, it seems like the overwhelming majority of the code simply
doesn't care. You could pick an order out of a hat and it would be
100% OK. But I haven't gone and looked through it with this specific
idea in mind.
> On the whole though I think it's probably a good idea to leave
> parent nodes in control of the timing, so I kind of side with
> your later comment about whether we want to change this at all.
My overall feeling here is that what Gather and Gather Merge is doing
is pretty weird. I think I kind of knew that at the time this was all
getting implemented and reviewed, but I wasn't keen to introduce more
infrastructure changes than necessary given that parallel query, as a
project, was still pretty new and I didn't want to give other hackers
more reasons to be unhappy with what was already a lot of very
wide-ranging change to the system. A good number of years having gone
by now, and other people having worked on that code some more, I'm not
too worried about someone calling for a wholesale revert of parallel
query. However, there's a second problem here as well, which is that
I'm still not sure what the right thing to do is. We've fiddled around
with the shutdown sequence for parallel query a number of times now,
and I think there's still stuff that doesn't work quite right,
especially around getting all of the instrumentation data back to the
leader. I haven't spent enough time on this recently enough to be sure
what if any problems remain, though.
So on the one hand, I don't really like the fact that we have an
ad-hoc recursion arrangement here, instead of using
planstate_tree_walker or, as Amit proposes, a List. Giving subordinate
nodes control over the ordering when they don't really need it just
means we have more code with more possibility for bugs and less
certainty about whether the theoretical flexibility is doing anything
in practice. But on the other hand, because we know that at least for
the Gather/GatherMerge case it seems like it probably matters
somewhat, it definitely seems appealing not to change anything as part
of this patch set that we don't really have to.
I've had it firmly in my mind here that we were going to need to
change something somehow -- I mean, the possibility of returning in
the middle of node initialization seems like a pretty major change to
the way this stuff works, and it seems hard for me to believe that we
can just do that and not have to adjust any code anywhere else. Can it
really be true that we can do that and yet not end up creating any
states anywhere with which the current cleanup code is unprepared to
cope? Maybe, but it would seem like rather good luck if that's how it
shakes out. Still, at the moment, I'm having a hard time understanding
what this particular change buys us.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-08 14:32 Amit Langote <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-08-08 14:32 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Tue, Aug 8, 2023 at 12:36 AM Robert Haas <[email protected]> wrote:
> On Thu, Aug 3, 2023 at 4:37 AM Amit Langote <[email protected]> wrote:
> > Here's a patch set where the refactoring to move the ExecutorStart()
> > calls to be closer to GetCachedPlan() (for the call sites that use a
> > CachedPlan) is extracted into a separate patch, 0002. Its commit
> > message notes an aspect of this refactoring that I feel a bit nervous
> > about -- needing to also move the CommandCounterIncrement() call from
> > the loop in PortalRunMulti() to PortalStart() which now does
> > ExecutorStart() for the PORTAL_MULTI_QUERY case.
>
> I spent some time today reviewing 0001. Here are a few thoughts and
> notes about things that I looked at.
Thanks for taking a look at this.
> First, I wondered whether it was really adequate for ExecEndPlan() to
> just loop over estate->es_plan_nodes and call it good. Put
> differently, is it possible that we could ever have more than one
> relevant EState, say for a subplan or an EPQ execution or something,
> so that this loop wouldn't cover everything? I found nothing to make
> me think that this is a real danger.
Check.
> Second, I wondered whether the ordering of cleanup operations could be
> an issue. Right now, a node can position cleanup code before, after,
> or both before and after recursing to child nodes, whereas with this
> design change, the cleanup code will always be run before recursing to
> child nodes.
Because a node is appended to es_planstate_nodes at the end of
ExecInitNode(), child nodes get added before their parent nodes. So
the children are cleaned up first.
> Here, I think we have problems. Both ExecGather and
> ExecEndGatherMerge intentionally clean up the children before the
> parent, so that the child shutdown happens before
> ExecParallelCleanup(). Based on the comment and commit
> acf555bc53acb589b5a2827e65d655fa8c9adee0, this appears to be
> intentional, and you can sort of see why from looking at the stuff
> that happens in ExecParallelCleanup(). If the instrumentation data
> vanishes before the child nodes have a chance to clean things up,
> maybe EXPLAIN ANALYZE won't reflect that instrumentation any more. If
> the DSA vanishes, maybe we'll crash if we try to access it. If we
> actually reach DestroyParallelContext(), we're just going to start
> killing the workers. None of that sounds like what we want.
>
> The good news, of a sort, is that I think this might be the only case
> of this sort of problem. Most nodes recurse at the end, after doing
> all the cleanup, so the behavior won't change. Moreover, even if it
> did, most cleanup operations look pretty localized -- they affect only
> the node itself, and not its children. A somewhat interesting case is
> nodes associated with subplans. Right now, because of the coding of
> ExecEndPlan, nodes associated with subplans are all cleaned up at the
> very end, after everything that's not inside of a subplan. But with
> this change, they'd get cleaned up in the order of initialization,
> which actually seems more natural, as long as it doesn't break
> anything, which I think it probably won't, since as I mention in most
> cases node cleanup looks quite localized, i.e. it doesn't care whether
> it happens before or after the cleanup of other nodes.
>
> I think something will have to be done about the parallel query stuff,
> though. I'm not sure exactly what. It is a little weird that Gather
> and Gather Merge treat starting and killing workers as a purely
> "private matter" that they can decide to handle without the executor
> overall being very much aware of it. So maybe there's a way that some
> of the cleanup logic here could be hoisted up into the general
> executor machinery, that is, first end all the nodes, and then go
> back, and end all the parallelism using, maybe, another list inside of
> the estate. However, I think that the existence of ExecShutdownNode()
> is a complication here -- we need to make sure that we don't break
> either the case where that happen before overall plan shutdown, or the
> case where it doesn't.
Given that children are closed before parent, the order of operations
in ExecEndGather[Merge] is unchanged.
> Third, a couple of minor comments on details of how you actually made
> these changes in the patch set. Personally, I would remove all of the
> "is closed separately" comments that you added. I think it's a
> violation of the general coding principle that you should make the
> code look like it's always been that way. Sure, in the immediate
> future, people might wonder why you don't need to recurse, but 5 or 10
> years from now that's just going to be clutter. Second, in the cases
> where the ExecEndNode functions end up completely empty, I would
> suggest just removing the functions entirely and making the switch
> that dispatches on the node type have a switch case that lists all the
> nodes that don't need a callback here and say /* Nothing do for these
> node types */ break;. This will save a few CPU cycles and I think it
> will be easier to read as well.
I agree with both suggestions.
> Fourth, I wonder whether we really need this patch at all. I initially
> thought we did, because if we abandon the initialization of a plan
> partway through, then we end up with a plan that is in a state that
> previously would never have occurred, and we still have to be able to
> clean it up. However, perhaps it's a difference without a distinction.
> Say we have a partial plan tree, where not all of the PlanState nodes
> ever got created. We then just call the existing version of
> ExecEndPlan() on it, with no changes. What goes wrong? Sure, we might
> call ExecEndNode() on some null pointers where in the current world
> there would always be valid pointers, but ExecEndNode() will handle
> that just fine, by doing nothing for those nodes, because it starts
> with a NULL-check.
Well, not all cleanup actions for a given node type are a recursive
call to ExecEndNode(), some are also things like this:
/*
* clean out the tuple table
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
But should ExecInitNode() subroutines return the partially initialized
PlanState node or NULL on detecting invalidation? If I'm
understanding how you think this should be working correctly, I think
you mean the former, because if it were the latter, ExecInitNode()
would end up returning NULL at the top for the root and then there's
nothing to pass to ExecEndNode(), so no way to clean up to begin with.
In that case, I think we will need to adjust ExecEndNode() subroutines
to add `if (node->ps.ps_ResultTupleSlot)` in the above code, for
example. That's something Tom had said he doesn't like very much [1].
Some node types such as Append, BitmapAnd, etc. that contain a list of
subplans would need some adjustment, such as using palloc0 for
as_appendplans[], etc. so that uninitialized subplans have NULL in the
array.
There are also issues around ForeignScan, CustomScan
ExecEndNode()-time callbacks when they are partially initialized -- is
it OK to call the *EndScan callback if the *BeginScan one may not have
been called to begin with? Though, perhaps we can adjust the
ExecInitNode() subroutines for those to return NULL by opening the
relation and checking for invalidation at the beginning instead of in
the middle. That should be done for all Scan or leaf-level node
types.
Anyway, I guess, for the patch's purpose, maybe we should bite the
bullet and make those adjustments rather than change ExecEndNode() as
proposed. I can give that another try.
> Another alternative design might be to switch ExecEndNode to use
> planstate_tree_walker to walk the node tree, removing the walk from
> the node-type-specific functions as in this patch, and deleting the
> end-node functions that are no longer required altogether, as proposed
> above. I somehow feel that this would be cleaner than the status quo,
> but here again, I'm not sure we really need it. planstate_tree_walker
> would just pass over any NULL pointers that it found without doing
> anything, but the current code does that too, so while this might be
> more beautiful than what we have now, I'm not sure that there's any
> real reason to do it. The fact that, like the current patch, it would
> change the order in which nodes are cleaned up is also an issue -- the
> Gather/Gather Merge ordering issues might be easier to handle this way
> with some hack in ExecEndNode() than they are with the design you have
> now, but we'd still have to do something about them, I believe.
It might be interesting to see if introducing planstate_tree_walker()
in ExecEndNode() makes it easier to reason about ExecEndNode()
generally speaking, but I think you may be that doing so may not
really make matters easier for the partially initialized planstate
tree case.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-08 16:05 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Robert Haas @ 2023-08-08 16:05 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Tue, Aug 8, 2023 at 10:32 AM Amit Langote <[email protected]> wrote:
> But should ExecInitNode() subroutines return the partially initialized
> PlanState node or NULL on detecting invalidation? If I'm
> understanding how you think this should be working correctly, I think
> you mean the former, because if it were the latter, ExecInitNode()
> would end up returning NULL at the top for the root and then there's
> nothing to pass to ExecEndNode(), so no way to clean up to begin with.
> In that case, I think we will need to adjust ExecEndNode() subroutines
> to add `if (node->ps.ps_ResultTupleSlot)` in the above code, for
> example. That's something Tom had said he doesn't like very much [1].
Yeah, I understood Tom's goal as being "don't return partially
initialized nodes."
Personally, I'm not sure that's an important goal. In fact, I don't
even think it's a desirable one. It doesn't look difficult to audit
the end-node functions for cases where they'd fail if a particular
pointer were NULL instead of pointing to some real data, and just
fixing all such cases to have NULL-tests looks like purely mechanical
work that we are unlikely to get wrong. And at least some cases
wouldn't require any changes at all.
If we don't do that, the complexity doesn't go away. It just moves
someplace else. Presumably what we do in that case is have
ExecInitNode functions undo any initialization that they've already
done before returning NULL. There are basically two ways to do that.
Option one is to add code at the point where they return early to
clean up anything they've already initialized, but that code is likely
to substantially duplicate whatever the ExecEndNode function already
knows how to do, and it's very easy for logic like this to get broken
if somebody rearranges an ExecInitNode function down the road. Option
two is to rearrange the ExecInitNode functions now, to open relations
or recurse at the beginning, so that we discover the need to fail
before we initialize anything. That restricts our ability to further
rearrange the functions in future somewhat, but more importantly,
IMHO, it introduces more risk right now. Checking that the ExecEndNode
function will not fail if some pointers are randomly null is a lot
easier than checking that changing the order of operations in an
ExecInitNode function breaks nothing.
I'm not here to say that we can't do one of those things. But I think
adding null-tests to ExecEndNode functions looks like *far* less work
and *way* less risk.
There's a second issue here, too, which is when we abort ExecInitNode
partway through, how do we signal that? You're rightly pointing out
here that if we do that by returning NULL, then we don't do it by
returning a pointer to the partially initialized node that we just
created, which means that we either need to store those partially
initialized nodes in a separate data structure as you propose to do in
0001, or else we need to pick a different signalling convention. We
could change (a) ExecInitNode to have an additional argument, bool
*kaboom, or (b) we could make it return bool and return the node
pointer via a new additional argument, or (c) we could put a Boolean
flag into the estate and let the function signal failure by flipping
the value of the flag. If we do any of those things, then as far as I
can see 0001 is unnecessary. If we do none of them but also avoid
creating partially initialized nodes by one of the two techniques
mentioned two paragraphs prior, then 0001 is also unnecessary. If we
do none of them but do create partially initialized nodes, then we
need 0001.
So if this were a restaurant menu, then it might look like this:
Prix Fixe Menu (choose one from each)
First Course - How do we clean up after partial initialization?
(1) ExecInitNode functions produce partially initialized nodes
(2) ExecInitNode functions get refactored so that the stuff that can
cause early exit always happens first, so that no cleanup is ever
needed
(3) ExecInitNode functions do any required cleanup in situ
Second Course - How do we signal that initialization stopped early?
(A) Return NULL.
(B) Add a bool * out-parmeter to ExecInitNode.
(C) Add a Node * out-parameter to ExecInitNode and change the return
value to bool.
(D) Add a bool to the EState.
(E) Something else, maybe.
I think that we need 0001 if we choose specifically (1) and (A). My
gut feeling is that the least-invasive way to do this project is to
choose (1) and (D). My second choice would be (1) and (C), and my
third choice would be (1) and (A). If I can't have (1), I think I
prefer (2) over (3), but I also believe I prefer hiding in a deep hole
to either of them. Maybe I'm not seeing the whole picture correctly
here, but both (2) and (3) look awfully painful to me.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-11 05:31 Amit Langote <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-08-11 05:31 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Wed, Aug 9, 2023 at 1:05 AM Robert Haas <[email protected]> wrote:
> On Tue, Aug 8, 2023 at 10:32 AM Amit Langote <[email protected]> wrote:
> > But should ExecInitNode() subroutines return the partially initialized
> > PlanState node or NULL on detecting invalidation? If I'm
> > understanding how you think this should be working correctly, I think
> > you mean the former, because if it were the latter, ExecInitNode()
> > would end up returning NULL at the top for the root and then there's
> > nothing to pass to ExecEndNode(), so no way to clean up to begin with.
> > In that case, I think we will need to adjust ExecEndNode() subroutines
> > to add `if (node->ps.ps_ResultTupleSlot)` in the above code, for
> > example. That's something Tom had said he doesn't like very much [1].
>
> Yeah, I understood Tom's goal as being "don't return partially
> initialized nodes."
>
> Personally, I'm not sure that's an important goal. In fact, I don't
> even think it's a desirable one. It doesn't look difficult to audit
> the end-node functions for cases where they'd fail if a particular
> pointer were NULL instead of pointing to some real data, and just
> fixing all such cases to have NULL-tests looks like purely mechanical
> work that we are unlikely to get wrong. And at least some cases
> wouldn't require any changes at all.
>
> If we don't do that, the complexity doesn't go away. It just moves
> someplace else. Presumably what we do in that case is have
> ExecInitNode functions undo any initialization that they've already
> done before returning NULL. There are basically two ways to do that.
> Option one is to add code at the point where they return early to
> clean up anything they've already initialized, but that code is likely
> to substantially duplicate whatever the ExecEndNode function already
> knows how to do, and it's very easy for logic like this to get broken
> if somebody rearranges an ExecInitNode function down the road.
Yeah, I too am not a fan of making ExecInitNode() clean up partially
initialized nodes.
> Option
> two is to rearrange the ExecInitNode functions now, to open relations
> or recurse at the beginning, so that we discover the need to fail
> before we initialize anything. That restricts our ability to further
> rearrange the functions in future somewhat, but more importantly,
> IMHO, it introduces more risk right now. Checking that the ExecEndNode
> function will not fail if some pointers are randomly null is a lot
> easier than checking that changing the order of operations in an
> ExecInitNode function breaks nothing.
>
> I'm not here to say that we can't do one of those things. But I think
> adding null-tests to ExecEndNode functions looks like *far* less work
> and *way* less risk.
+1
> There's a second issue here, too, which is when we abort ExecInitNode
> partway through, how do we signal that? You're rightly pointing out
> here that if we do that by returning NULL, then we don't do it by
> returning a pointer to the partially initialized node that we just
> created, which means that we either need to store those partially
> initialized nodes in a separate data structure as you propose to do in
> 0001,
>
> or else we need to pick a different signalling convention. We
> could change (a) ExecInitNode to have an additional argument, bool
> *kaboom, or (b) we could make it return bool and return the node
> pointer via a new additional argument, or (c) we could put a Boolean
> flag into the estate and let the function signal failure by flipping
> the value of the flag.
The failure can already be detected by seeing that
ExecPlanIsValid(estate) is false. The question is what ExecInitNode()
or any of its subroutines should return once it is. I think the
following convention works:
Return partially initialized state from ExecInit* function where we
detect the invalidation after calling ExecInitNode() on a child plan,
so that ExecEndNode() can recurse to clean it up.
Return NULL from ExecInit* functions where we detect the invalidation
after opening and locking a relation but before calling ExecInitNode()
to initialize a child plan if there's one at all. Even if we may set
things like ExprContext, TupleTableSlot fields, they are cleaned up
independently of the plan tree anyway via the cleanup called with
es_exprcontexts, es_tupleTable, respectively. I even noticed bits
like this in ExecEnd* functions:
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->ioss_RuntimeContext)
- FreeExprContext(node->ioss_RuntimeContext, true);
-#endif
So, AFAICS, ExprContext, TupleTableSlot cleanup in ExecNode* functions
is unnecessary but remain around because nobody cared about and got
around to getting rid of it.
> If we do any of those things, then as far as I
> can see 0001 is unnecessary. If we do none of them but also avoid
> creating partially initialized nodes by one of the two techniques
> mentioned two paragraphs prior, then 0001 is also unnecessary. If we
> do none of them but do create partially initialized nodes, then we
> need 0001.
>
> So if this were a restaurant menu, then it might look like this:
>
> Prix Fixe Menu (choose one from each)
>
> First Course - How do we clean up after partial initialization?
> (1) ExecInitNode functions produce partially initialized nodes
> (2) ExecInitNode functions get refactored so that the stuff that can
> cause early exit always happens first, so that no cleanup is ever
> needed
> (3) ExecInitNode functions do any required cleanup in situ
>
> Second Course - How do we signal that initialization stopped early?
> (A) Return NULL.
> (B) Add a bool * out-parmeter to ExecInitNode.
> (C) Add a Node * out-parameter to ExecInitNode and change the return
> value to bool.
> (D) Add a bool to the EState.
> (E) Something else, maybe.
>
> I think that we need 0001 if we choose specifically (1) and (A). My
> gut feeling is that the least-invasive way to do this project is to
> choose (1) and (D). My second choice would be (1) and (C), and my
> third choice would be (1) and (A). If I can't have (1), I think I
> prefer (2) over (3), but I also believe I prefer hiding in a deep hole
> to either of them. Maybe I'm not seeing the whole picture correctly
> here, but both (2) and (3) look awfully painful to me.
I think what I've ended up with in the attached 0001 (WIP) is both
(1), (2), and (D). As mentioned above, (D) is implemented with the
ExecPlanStillValid() function.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v45-0006-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 2-v45-0006-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 77a07e115a18527ebe9312d703d5c76d94fc1f84 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v45 6/6] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 1197ff2bf2..d678940a3f 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1640,12 +1640,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index af92d2b3c3..f0320cfa34 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -837,6 +837,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 719a728319..06bf829b78 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v45-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 3-v45-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 0a9161783a415ea5593514cb4d650cd3bc6c0601 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v45 3/6] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..8ac1d3909b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v45-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch (70.1K, 4-v45-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 0ed0a6928f0c9a4c713850b6f0d72ab5a00c8425 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v45 5/6] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Also,
ExecutorStart() now returns true or false to indicate whether or not
QueryDesc.planstate points to a successfully initialized PlanState
tree. Call sites that use GetCachedPlan() to get the plan trees to
pass to the executor should now be prepared to retry in the cases
where ExecutorStart() returns false.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 7 +-
src/backend/commands/createas.c | 10 +-
src/backend/commands/explain.c | 33 +++-
src/backend/commands/extension.c | 4 +-
src/backend/commands/matview.c | 10 +-
src/backend/commands/portalcmds.c | 5 +-
src/backend/commands/prepare.c | 23 ++-
src/backend/executor/README | 39 ++++-
src/backend/executor/execMain.c | 64 +++++--
src/backend/executor/execParallel.c | 14 +-
src/backend/executor/execPartition.c | 10 ++
src/backend/executor/execUtils.c | 61 +++++--
src/backend/executor/functions.c | 5 +-
src/backend/executor/nodeAppend.c | 19 +++
src/backend/executor/nodeMergeAppend.c | 19 +++
src/backend/executor/spi.c | 26 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++++
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 49 +++++-
src/backend/utils/cache/lsyscache.c | 21 +++
src/backend/utils/cache/plancache.c | 156 +++++++-----------
src/include/commands/explain.h | 3 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 7 +-
src/include/storage/lmgr.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/lsyscache.h | 1 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 156 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
33 files changed, 786 insertions(+), 181 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 55b957d251..1160a7326a 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -963,13 +963,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -992,6 +994,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..916d6dced3 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -567,8 +568,10 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..e5cce4c07c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,12 +325,16 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 59d57f9c10..6171a20fe2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -416,7 +416,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
- queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
params, queryEnv);
Assert(queryDesc);
@@ -429,9 +429,11 @@ ExplainOneQuery(Query *query, int cursorOptions,
/*
* ExplainQueryDesc
* Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
*/
QueryDesc *
-ExplainQueryDesc(PlannedStmt *stmt,
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv)
{
@@ -467,7 +469,7 @@ ExplainQueryDesc(PlannedStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(stmt, queryString,
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
@@ -481,8 +483,18 @@ ExplainQueryDesc(PlannedStmt *stmt,
if (into)
eflags |= GetIntoRelEFlags(into);
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, eflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
return queryDesc;
}
@@ -4884,6 +4896,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..93a683e312 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..38795ce7ca 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,16 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..5120f93414 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,10 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 1e9a98ad6e..156c3c5fee 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if the cached plan was found to have been invalidated
+ * when initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -642,9 +650,14 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
QueryDesc *queryDesc;
- queryDesc = ExplainQueryDesc(pstmt, queryString,
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
into, es, paramLI, queryEnv);
- Assert(queryDesc != NULL);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
queryEnv, &planduration,
(es->buffers ? &bufusage : NULL));
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..0a7bb42ccb 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,37 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to execution-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree after execution-initialization-pruning has been
+performed. It does that by looking at CachedPlan.is_valid of the CachedPlan
+passed to it. If the plan tree is indeed stale (is_valid=false), the executor
+must give up continuing to initialize it any further and return to the caller
+letting it know that the execution must be retried with a new plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +347,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 1a848b1c20..1197ff2bf2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,11 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -620,6 +630,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,9 +850,12 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -850,12 +874,12 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
- estate->es_cachedplan = NULL;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -1016,7 +1040,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
queryDesc->tupDesc = tupType;
Assert(planstate != NULL);
queryDesc->planstate = planstate;
- return;
+ return true;
plan_init_suspended:
/*
@@ -1024,6 +1048,7 @@ plan_init_suspended:
* will clean up initialized plan nodes from estate->es_planstate_nodes.
*/
queryDesc->planstate = planstate;
+ return false;
}
/*
@@ -1441,7 +1466,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2873,7 +2898,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2960,6 +2986,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -3003,6 +3035,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..bfa2a8ec18 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1430,7 +1439,8 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index e88455368c..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f4611bdd27..af92d2b3c3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -804,7 +804,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -817,15 +835,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -833,6 +842,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..4ddf4fd7a9 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -862,7 +863,9 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 5d9fa4bff3..ce60ca2126 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,25 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 255b05aad3..657fa91ec4 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,25 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index d36ca35d3a..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2669,6 +2680,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
@@ -2682,10 +2694,16 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
else
eflags = EXEC_FLAG_SKIP_TRIGGERS;
- ExecutorStart(qdesc, eflags);
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
-
FreeQueryDesc(qdesc);
}
else
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..88724a8d67 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 701808f303..48cd6f4304 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -60,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -72,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -341,10 +343,12 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
@@ -353,6 +357,7 @@ PortalStart(Portal portal, ParamListInfo params,
MemoryContext oldContext;
QueryDesc *queryDesc;
int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -407,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -431,8 +437,19 @@ PortalStart(Portal portal, ParamListInfo params,
else
myeflags = eflags;
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
* This tells PortalCleanup to shut down the executor, though
@@ -525,7 +542,7 @@ PortalStart(Portal portal, ParamListInfo params,
* Create the QueryDesc. DestReceiver will be set in
* PortalRunMulti() before calling ExecutorRun().
*/
- queryDesc = CreateQueryDesc(plan,
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -541,7 +558,20 @@ PortalStart(Portal portal, ParamListInfo params,
if (is_utility)
continue;
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
PopActiveSnapshot();
}
}
@@ -563,12 +593,15 @@ PortalStart(Portal portal, ParamListInfo params,
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index d67cd9a405..c5a7616b33 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,15 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan contains any child relations that would have been added by the
+ * planner, they would not have been locked yet, because AcquirePlannerLocks()
+ * only locks relations that would be present in the original query's range
+ * table (that is, before entering the planner). So, the plan could go stale
+ * before it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables during the plan
+ * initialization phase, and if it is not, ask the caller to recreate the
+ * plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, because they will not have been locked as here we only lock the
+ * tables mentioned in the original query. Inheritance/partition child tables
+ * are locked by the executor when initializing the plan tree and if the plan
+ * gets invalidated as a result of taking those locks, the executor must ask
+ * the caller to get a new plan by calling here again. Locking of the child
+ * tables is deferred to the executor in this manner, because not all child
+ * tables may need to be locked as some may get pruned during the executor
+ * plan initialization which performs initial pruing on any nodes that
+ * support partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1164,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1362,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1739,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 08ea852b65..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,7 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv);
extern void ExplainOnePlan(QueryDesc *queryDesc,
@@ -108,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 8ec636cab8..edf2f13d04 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
@@ -602,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v45-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 5-v45-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From af264e901e7636bf081a68257544d933b16d77ca Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v45 4/6] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fe003ded50..72f2b0c04f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v45-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch (25.8K, 6-v45-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch)
download | inline diff:
From 1b27fcbdfed5426ff19721d3e19664adb84f9c7d Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 3 Aug 2023 12:34:31 +0900
Subject: [PATCH v45 2/6] Refactoring to move ExecutorStart() calls to be near
GetCachedPlan()
An upcoming patch will make ExecutorStart() detect the invalidation
of a CachedPlan when initializing the plan tree contained in it. A
caller must retry with a new CachedPlan when ExecutorStart() detects
an invalidation. Having the ExecutorStart() in the same or nearby
as GetCachedPlan() makes it more convenient to implement the replan
loop.
The following sites have thus been modified:
* The ExecutorStart() call in ExplainOnePlan() is moved, along with
CreateQueryDesc(), into a new function ExplainQueryDesc(), which its
callers now call before calling it.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and the associated
memory context field. One unintended consequence is that the
CommandCounterIncrement() between queries in PORTAL_MULTI_QUERY
cases is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still seems to work because the Snapshot
registered in QueryDesc/EState is updated to account for the
CCI().
---
src/backend/commands/explain.c | 121 ++++++-----
src/backend/commands/prepare.c | 12 +-
src/backend/executor/spi.c | 27 +--
src/backend/tcop/pquery.c | 311 +++++++++++++----------------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 6 +-
src/include/utils/portal.h | 2 +
7 files changed, 250 insertions(+), 238 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..59d57f9c10 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,77 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /* Call ExecutorStart to prepare the plan for execution. */
+ ExecutorStart(queryDesc, eflags);
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +590,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
- Assert(plannedstmt->commandType != CMD_UTILITY);
-
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +608,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..1e9a98ad6e 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -639,8 +639,16 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, queryString,
+ into, es, paramLI, queryEnv);
+ Assert(queryDesc != NULL);
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..d36ca35d3a 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -2661,6 +2661,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2674,8 +2675,17 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
+
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2860,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2906,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..701808f303 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -116,86 +111,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -435,10 +350,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +362,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +384,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -489,8 +403,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
portal->sourceText,
@@ -501,30 +415,41 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
+ /* Call ExecutorStart to prepare the plan for execution. */
ExecutorStart(queryDesc, myeflags);
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +461,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +483,69 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,7 +558,6 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
@@ -604,7 +567,6 @@ PortalStart(Portal portal, ParamListInfo params,
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
portal->status = PORTAL_READY;
}
@@ -1193,7 +1155,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1176,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1196,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1224,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1311,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..08ea852b65 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v45-0001-Add-support-for-allowing-ExecInitNode-to-detect-.patch (52.4K, 7-v45-0001-Add-support-for-allowing-ExecInitNode-to-detect-.patch)
download | inline diff:
From ddaa761e96cd4b34035edd40f6bc17b953574496 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 11 Aug 2023 14:09:29 +0900
Subject: [PATCH v45 1/6] Add support for allowing ExecInitNode to detect
CachedPlan invalidation
This means ExecInitNode() will check for CachedPlan invalidation at
various points (after locking tables, initializing child plans, etc.)
and return either a partially initialized planstate node or NULL if
no cleanup is necessary.
ExecEndNode() subroutines now always check a pointer for nullness
before calling cleanup on it.
---
contrib/postgres_fdw/postgres_fdw.c | 4 ++++
src/backend/executor/execMain.c | 23 +++++++++++++++++--
src/backend/executor/execPartition.c | 4 ++++
src/backend/executor/execProcnode.c | 12 +++++++++-
src/backend/executor/execUtils.c | 2 ++
src/backend/executor/nodeAgg.c | 15 ++++--------
src/backend/executor/nodeAppend.c | 10 +++++---
src/backend/executor/nodeBitmapAnd.c | 7 +++---
src/backend/executor/nodeBitmapHeapscan.c | 19 +++++----------
src/backend/executor/nodeBitmapIndexscan.c | 10 ++------
src/backend/executor/nodeBitmapOr.c | 7 +++---
src/backend/executor/nodeCtescan.c | 12 ----------
src/backend/executor/nodeCustom.c | 16 ++++++-------
src/backend/executor/nodeForeignscan.c | 12 ++++------
src/backend/executor/nodeFunctionscan.c | 12 ----------
src/backend/executor/nodeGather.c | 6 ++---
src/backend/executor/nodeGatherMerge.c | 5 ++--
src/backend/executor/nodeGroup.c | 7 ++----
src/backend/executor/nodeHash.c | 7 ++----
src/backend/executor/nodeHashjoin.c | 16 ++++---------
src/backend/executor/nodeIncrementalSort.c | 10 ++------
src/backend/executor/nodeIndexonlyscan.c | 20 ++++------------
src/backend/executor/nodeIndexscan.c | 20 ++++------------
src/backend/executor/nodeLimit.c | 3 ++-
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 7 ++----
src/backend/executor/nodeMemoize.c | 12 +++-------
src/backend/executor/nodeMergeAppend.c | 6 ++++-
src/backend/executor/nodeMergejoin.c | 16 ++++---------
src/backend/executor/nodeModifyTable.c | 18 ++++++---------
.../executor/nodeNamedtuplestorescan.c | 11 ---------
src/backend/executor/nodeNestloop.c | 15 ++++--------
src/backend/executor/nodeProjectSet.c | 12 ++--------
src/backend/executor/nodeRecursiveunion.c | 10 ++++++--
src/backend/executor/nodeResult.c | 12 ++--------
src/backend/executor/nodeSamplescan.c | 16 +++----------
src/backend/executor/nodeSeqscan.c | 14 ++---------
src/backend/executor/nodeSetOp.c | 6 ++---
src/backend/executor/nodeSort.c | 9 ++------
src/backend/executor/nodeSubqueryscan.c | 14 ++---------
src/backend/executor/nodeTableFuncscan.c | 12 ----------
src/backend/executor/nodeTidrangescan.c | 14 ++---------
src/backend/executor/nodeTidscan.c | 14 ++---------
src/backend/executor/nodeUnique.c | 7 ++----
src/backend/executor/nodeValuesscan.c | 13 -----------
src/backend/executor/nodeWindowAgg.c | 13 ++---------
src/backend/executor/nodeWorktablescan.c | 11 ---------
src/include/executor/executor.h | 12 ++++++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 +++++++++++
50 files changed, 191 insertions(+), 360 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..1a848b1c20 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -839,7 +839,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -855,6 +855,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = NULL;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +887,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,9 +956,10 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
-
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
i++;
}
@@ -966,6 +970,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,6 +1014,15 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
+ queryDesc->planstate = planstate;
+ return;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_planstate_nodes.
+ */
queryDesc->planstate = planstate;
}
@@ -3010,6 +3025,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..e88455368c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1801,6 +1801,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1929,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..842c6751c5 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,13 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +394,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ return result;
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..f4611bdd27 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -848,6 +848,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..f46c3df199 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3150,7 +3150,8 @@ hashagg_reset_spill_state(AggState *aggstate)
}
/* free batches */
- list_free_deep(aggstate->hash_batches);
+ if (aggstate->hash_batches)
+ list_free_deep(aggstate->hash_batches);
aggstate->hash_batches = NIL;
/* close tape set */
@@ -3304,6 +3305,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return aggstate;
/*
* initialize source tuple type.
@@ -4357,16 +4360,6 @@ ExecEndAgg(AggState *node)
if (node->hashcontext)
ReScanExprContext(node->hashcontext);
- /*
- * We don't actually free any ExprContexts here (see comment in
- * ExecFreeExprContext), just unlinking the output one from the plan node
- * suffices.
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..5d9fa4bff3 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -147,6 +147,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -185,8 +187,9 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ appendstate->appendplans = appendplanstates =
+ (PlanState **) palloc0(nplans * sizeof(PlanState *));
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +224,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..93e5de0c1a 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -78,7 +78,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
bitmapandstate->ps.state = estate;
bitmapandstate->ps.ExecProcNode = ExecBitmapAnd;
bitmapandstate->bitmapplans = bitmapplanstates;
- bitmapandstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -88,8 +87,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmapandstate;
+ bitmapandstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..3cdece852c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -655,18 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subplans
*/
@@ -693,7 +681,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
}
/* ----------------------------------------------------------------
@@ -763,11 +752,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..4200472d02 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -184,14 +184,6 @@ ExecEndBitmapIndexScan(BitmapIndexScanState *node)
indexRelationDesc = node->biss_RelationDesc;
indexScanDesc = node->biss_ScanDesc;
- /*
- * Free the exprcontext ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- if (node->biss_RuntimeContext)
- FreeExprContext(node->biss_RuntimeContext, true);
-#endif
-
/*
* close the index relation (no-op if we didn't open it)
*/
@@ -263,6 +255,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return indexstate;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..e0e9228e35 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -79,7 +79,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
bitmaporstate->ps.state = estate;
bitmaporstate->ps.ExecProcNode = ExecBitmapOr;
bitmaporstate->bitmapplans = bitmapplanstates;
- bitmaporstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -89,8 +88,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmaporstate;
+ bitmaporstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index cc4c4243e2..a0c0c4be33 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -287,18 +287,6 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
void
ExecEndCteScan(CteScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* If I am the leader, free the tuplestore.
*/
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..38061c30b9 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
@@ -127,15 +129,11 @@ ExecCustomScan(PlanState *pstate)
void
ExecEndCustomScan(CustomScanState *node)
{
- Assert(node->methods->EndCustomScan != NULL);
- node->methods->EndCustomScan(node);
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* Clean out the tuple table */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
+ if (node->methods)
+ {
+ Assert(node->methods->EndCustomScan != NULL);
+ node->methods->EndCustomScan(node);
+ }
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..a3705082a9 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
@@ -312,14 +316,6 @@ ExecEndForeignScan(ForeignScanState *node)
/* Shut down any outer plan. */
if (outerPlanState(node))
ExecEndNode(outerPlanState(node));
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean out the tuple table */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index dd06ef8aee..a49c1a2c85 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -523,18 +523,6 @@ ExecEndFunctionScan(FunctionScanState *node)
{
int i;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release slots and tuplestore resources
*/
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..6b26e03f74 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gatherstate;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
@@ -250,9 +253,6 @@ ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..84412f94bb 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
@@ -290,9 +292,6 @@ ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..b6068887f6 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return grpstate;
/*
* Initialize scan slot and type.
@@ -228,11 +230,6 @@ ExecEndGroup(GroupState *node)
{
PlanState *outerPlan;
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..030bf0ed43 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
@@ -415,11 +417,6 @@ ExecEndHash(HashState *node)
{
PlanState *outerPlan;
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..49a6ba4276 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
@@ -867,18 +871,6 @@ ExecEndHashJoin(HashJoinState *node)
node->hj_HashTable = NULL;
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->hj_OuterTupleSlot);
- ExecClearTuple(node->hj_HashTupleSlot);
-
/*
* clean up subtrees
*/
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..6caa1aa306 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return incrsortstate;
/*
* Initialize scan slot and type.
@@ -1079,14 +1081,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- /* clean out the scan tuple */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- /* must drop standalone tuple slots from outer node */
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
-
/*
* Release tuplesort resources.
*/
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea7fd89c0c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -380,22 +380,6 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->ioss_RuntimeContext)
- FreeExprContext(node->ioss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
@@ -512,6 +496,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -565,6 +551,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..906358011a 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -794,22 +794,6 @@ ExecEndIndexScan(IndexScanState *node)
indexRelationDesc = node->iss_RelationDesc;
indexScanDesc = node->iss_ScanDesc;
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->iss_RuntimeContext)
- FreeExprContext(node->iss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
@@ -925,6 +909,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -970,6 +956,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..6760de0f25 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return limitstate;
/*
* initialize child expressions
@@ -534,7 +536,6 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
void
ExecEndLimit(LimitState *node)
{
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..2599332f01 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..b974ebdc8a 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
@@ -239,11 +241,6 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
void
ExecEndMaterial(MaterialState *node)
{
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..d0cdbe1fd7 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
@@ -1043,6 +1045,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable)
{
int count;
uint64 mem = 0;
@@ -1091,15 +1094,6 @@ ExecEndMemoize(MemoizeState *node)
/* Remove the cache context */
MemoryContextDelete(node->tableContext);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to cache result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..255b05aad3 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -95,6 +95,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -120,7 +122,7 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +153,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..e7f4512419 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
@@ -1642,18 +1646,6 @@ ExecEndMergeJoin(MergeJoinState *node)
{
MJ1_printf("ExecEndMergeJoin: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->mj_MarkedTupleSlot);
-
/*
* shut down the subplans
*/
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5005d8c0d1..c28d5058e9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3985,6 +3985,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4012,6 +4015,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4039,6 +4044,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
@@ -4446,17 +4453,6 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* Terminate EPQ execution if active
*/
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index 46832ad82f..e142ef593b 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -164,17 +164,6 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
void
ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..0158a3e592 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
/*
* Initialize result slot, type and projection.
@@ -363,17 +367,6 @@ ExecEndNestLoop(NestLoopState *node)
{
NL1_printf("ExecEndNestLoop: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..1b4774d4f7 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return state;
/*
* we don't use inner plan
@@ -320,16 +322,6 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
void
ExecEndProjectSet(ProjectSetState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..ca4f78685d 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
@@ -272,8 +276,10 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table)
+ tuplestore_end(node->working_table);
+ if (node->intermediate_table)
+ tuplestore_end(node->intermediate_table);
/* free subsidiary stuff including hashtable */
if (node->tempContext)
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..d4ea101cbe 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return resstate;
/*
* we don't use inner plan
@@ -240,16 +242,6 @@ ExecInitResult(Result *node, EState *estate, int eflags)
void
ExecEndResult(ResultState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..02a7db96e1 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
@@ -185,21 +187,9 @@ ExecEndSampleScan(SampleScanState *node)
/*
* Tell sampling function that we finished the scan.
*/
- if (node->tsmroutine->EndSampleScan)
+ if (node->tsmroutine && node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..48e20aa735 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
@@ -190,18 +192,6 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..7a3a142204 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
@@ -582,13 +584,9 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
void
ExecEndSetOp(SetOpState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/* free subsidiary stuff including hashtable */
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..3ebbc46604 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return sortstate;
/*
* Initialize scan slot and type.
@@ -303,13 +305,6 @@ ExecEndSort(SortState *node)
SO1_printf("ExecEndSort: %s\n",
"shutting down sort node");
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
/*
* Release tuplesort resources
*/
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..3c5c7c2ebb 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
@@ -167,18 +169,6 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
void
ExecEndSubqueryScan(SubqueryScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the upper tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subquery
*/
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 791cbd2372..a60dcd4943 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -213,18 +213,6 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
void
ExecEndTableFuncScan(TableFuncScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..d337f3d54a 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -331,18 +331,6 @@ ExecEndTidRangeScan(TidRangeScanState *node)
if (scan != NULL)
table_endscan(scan);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
@@ -386,6 +374,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..9637f354b2 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -472,18 +472,6 @@ ExecEndTidScan(TidScanState *node)
{
if (node->ss.ss_currentScanDesc)
table_endscan(node->ss.ss_currentScanDesc);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
@@ -529,6 +517,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..28630e380e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
@@ -168,11 +170,6 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
void
ExecEndUnique(UniqueState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- ExecFreeExprContext(&node->ps);
-
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index 32ace63017..3f86783ad7 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -328,19 +328,6 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
void
ExecEndValuesScan(ValuesScanState *node)
{
- /*
- * Free both exprcontexts
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->rowcontext;
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..a4153be495 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
@@ -2691,17 +2693,6 @@ ExecEndWindowAgg(WindowAggState *node)
ExecClearTuple(node->agg_row_slot);
ExecClearTuple(node->temp_slot_1);
ExecClearTuple(node->temp_slot_2);
- if (node->framehead_slot)
- ExecClearTuple(node->framehead_slot);
- if (node->frametail_slot)
- ExecClearTuple(node->frametail_slot);
-
- /*
- * Free both the expr contexts.
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->tmpcontext;
- ExecFreeExprContext(&node->ss.ps);
for (i = 0; i < node->numaggs; i++)
{
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 0c13448236..cc63ddfeca 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -190,17 +190,6 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
void
ExecEndWorkTableScan(WorkTableScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..8ec636cab8 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..719a728319 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from,
+ * one or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..c83a67fea3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-11 13:50 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-08-11 13:50 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Fri, Aug 11, 2023 at 14:31 Amit Langote <[email protected]> wrote:
> On Wed, Aug 9, 2023 at 1:05 AM Robert Haas <[email protected]> wrote:
> > On Tue, Aug 8, 2023 at 10:32 AM Amit Langote <[email protected]>
> wrote:
> > > But should ExecInitNode() subroutines return the partially initialized
> > > PlanState node or NULL on detecting invalidation? If I'm
> > > understanding how you think this should be working correctly, I think
> > > you mean the former, because if it were the latter, ExecInitNode()
> > > would end up returning NULL at the top for the root and then there's
> > > nothing to pass to ExecEndNode(), so no way to clean up to begin with.
> > > In that case, I think we will need to adjust ExecEndNode() subroutines
> > > to add `if (node->ps.ps_ResultTupleSlot)` in the above code, for
> > > example. That's something Tom had said he doesn't like very much [1].
> >
> > Yeah, I understood Tom's goal as being "don't return partially
> > initialized nodes."
> >
> > Personally, I'm not sure that's an important goal. In fact, I don't
> > even think it's a desirable one. It doesn't look difficult to audit
> > the end-node functions for cases where they'd fail if a particular
> > pointer were NULL instead of pointing to some real data, and just
> > fixing all such cases to have NULL-tests looks like purely mechanical
> > work that we are unlikely to get wrong. And at least some cases
> > wouldn't require any changes at all.
> >
> > If we don't do that, the complexity doesn't go away. It just moves
> > someplace else. Presumably what we do in that case is have
> > ExecInitNode functions undo any initialization that they've already
> > done before returning NULL. There are basically two ways to do that.
> > Option one is to add code at the point where they return early to
> > clean up anything they've already initialized, but that code is likely
> > to substantially duplicate whatever the ExecEndNode function already
> > knows how to do, and it's very easy for logic like this to get broken
> > if somebody rearranges an ExecInitNode function down the road.
>
> Yeah, I too am not a fan of making ExecInitNode() clean up partially
> initialized nodes.
>
> > Option
> > two is to rearrange the ExecInitNode functions now, to open relations
> > or recurse at the beginning, so that we discover the need to fail
> > before we initialize anything. That restricts our ability to further
> > rearrange the functions in future somewhat, but more importantly,
> > IMHO, it introduces more risk right now. Checking that the ExecEndNode
> > function will not fail if some pointers are randomly null is a lot
> > easier than checking that changing the order of operations in an
> > ExecInitNode function breaks nothing.
> >
> > I'm not here to say that we can't do one of those things. But I think
> > adding null-tests to ExecEndNode functions looks like *far* less work
> > and *way* less risk.
>
> +1
>
> > There's a second issue here, too, which is when we abort ExecInitNode
> > partway through, how do we signal that? You're rightly pointing out
> > here that if we do that by returning NULL, then we don't do it by
> > returning a pointer to the partially initialized node that we just
> > created, which means that we either need to store those partially
> > initialized nodes in a separate data structure as you propose to do in
> > 0001,
> >
> > or else we need to pick a different signalling convention. We
> > could change (a) ExecInitNode to have an additional argument, bool
> > *kaboom, or (b) we could make it return bool and return the node
> > pointer via a new additional argument, or (c) we could put a Boolean
> > flag into the estate and let the function signal failure by flipping
> > the value of the flag.
>
> The failure can already be detected by seeing that
> ExecPlanIsValid(estate) is false. The question is what ExecInitNode()
> or any of its subroutines should return once it is. I think the
> following convention works:
>
> Return partially initialized state from ExecInit* function where we
> detect the invalidation after calling ExecInitNode() on a child plan,
> so that ExecEndNode() can recurse to clean it up.
>
> Return NULL from ExecInit* functions where we detect the invalidation
> after opening and locking a relation but before calling ExecInitNode()
> to initialize a child plan if there's one at all. Even if we may set
> things like ExprContext, TupleTableSlot fields, they are cleaned up
> independently of the plan tree anyway via the cleanup called with
> es_exprcontexts, es_tupleTable, respectively. I even noticed bits
> like this in ExecEnd* functions:
>
> - /*
> - * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
> - */
> -#ifdef NOT_USED
> - ExecFreeExprContext(&node->ss.ps);
> - if (node->ioss_RuntimeContext)
> - FreeExprContext(node->ioss_RuntimeContext, true);
> -#endif
>
> So, AFAICS, ExprContext, TupleTableSlot cleanup in ExecNode* functions
> is unnecessary but remain around because nobody cared about and got
> around to getting rid of it.
>
> > If we do any of those things, then as far as I
> > can see 0001 is unnecessary. If we do none of them but also avoid
> > creating partially initialized nodes by one of the two techniques
> > mentioned two paragraphs prior, then 0001 is also unnecessary. If we
> > do none of them but do create partially initialized nodes, then we
> > need 0001.
> >
> > So if this were a restaurant menu, then it might look like this:
> >
> > Prix Fixe Menu (choose one from each)
> >
> > First Course - How do we clean up after partial initialization?
> > (1) ExecInitNode functions produce partially initialized nodes
> > (2) ExecInitNode functions get refactored so that the stuff that can
> > cause early exit always happens first, so that no cleanup is ever
> > needed
> > (3) ExecInitNode functions do any required cleanup in situ
> >
> > Second Course - How do we signal that initialization stopped early?
> > (A) Return NULL.
> > (B) Add a bool * out-parmeter to ExecInitNode.
> > (C) Add a Node * out-parameter to ExecInitNode and change the return
> > value to bool.
> > (D) Add a bool to the EState.
> > (E) Something else, maybe.
> >
> > I think that we need 0001 if we choose specifically (1) and (A). My
> > gut feeling is that the least-invasive way to do this project is to
> > choose (1) and (D). My second choice would be (1) and (C), and my
> > third choice would be (1) and (A). If I can't have (1), I think I
> > prefer (2) over (3), but I also believe I prefer hiding in a deep hole
> > to either of them. Maybe I'm not seeing the whole picture correctly
> > here, but both (2) and (3) look awfully painful to me.
>
> I think what I've ended up with in the attached 0001 (WIP) is both
> (1), (2), and (D). As mentioned above, (D) is implemented with the
> ExecPlanStillValid() function.
After removing the unnecessary cleanup code from most node types’ ExecEnd*
functions, one thing I’m tempted to do is remove the functions that do
nothing else but recurse to close the outerPlan, innerPlan child nodes. We
could instead have ExecEndNode() itself recurse to close outerPlan,
innerPlan child nodes at the top, which preserves the
close-child-before-self behavior for Gather* nodes, and close node type
specific cleanup functions for nodes that do have any local cleanup to do.
Perhaps, we could even use planstate_tree_walker() called at the top
instead of the usual bottom so that nodes with a list of child subplans
like Append also don’t need to have their own ExecEnd* functions.
> --
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-08-28 13:43 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Robert Haas @ 2023-08-28 13:43 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Fri, Aug 11, 2023 at 9:50 AM Amit Langote <[email protected]> wrote:
> After removing the unnecessary cleanup code from most node types’ ExecEnd* functions, one thing I’m tempted to do is remove the functions that do nothing else but recurse to close the outerPlan, innerPlan child nodes. We could instead have ExecEndNode() itself recurse to close outerPlan, innerPlan child nodes at the top, which preserves the close-child-before-self behavior for Gather* nodes, and close node type specific cleanup functions for nodes that do have any local cleanup to do. Perhaps, we could even use planstate_tree_walker() called at the top instead of the usual bottom so that nodes with a list of child subplans like Append also don’t need to have their own ExecEnd* functions.
I think 0001 needs to be split up. Like, this is code cleanup:
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
This is providing for NULL pointers where we don't currently:
- list_free_deep(aggstate->hash_batches);
+ if (aggstate->hash_batches)
+ list_free_deep(aggstate->hash_batches);
And this is the early return mechanism per se:
+ if (!ExecPlanStillValid(estate))
+ return aggstate;
I think at least those 3 kinds of changes deserve to be in separate
patches with separate commit messages explaining the rationale behind
each e.g. "Remove unnecessary cleanup calls in ExecEnd* functions.
These calls are no longer required, because <reasons>. Removing them
saves a few CPU cycles and simplifies planned refactoring, so do
that."
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-05 07:13 Amit Langote <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-09-05 07:13 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
Thanks for taking a look.
On Mon, Aug 28, 2023 at 10:43 PM Robert Haas <[email protected]> wrote:
> On Fri, Aug 11, 2023 at 9:50 AM Amit Langote <[email protected]> wrote:
> > After removing the unnecessary cleanup code from most node types’ ExecEnd* functions, one thing I’m tempted to do is remove the functions that do nothing else but recurse to close the outerPlan, innerPlan child nodes. We could instead have ExecEndNode() itself recurse to close outerPlan, innerPlan child nodes at the top, which preserves the close-child-before-self behavior for Gather* nodes, and close node type specific cleanup functions for nodes that do have any local cleanup to do. Perhaps, we could even use planstate_tree_walker() called at the top instead of the usual bottom so that nodes with a list of child subplans like Append also don’t need to have their own ExecEnd* functions.
>
> I think 0001 needs to be split up. Like, this is code cleanup:
>
> - /*
> - * Free the exprcontext
> - */
> - ExecFreeExprContext(&node->ss.ps);
>
> This is providing for NULL pointers where we don't currently:
>
> - list_free_deep(aggstate->hash_batches);
> + if (aggstate->hash_batches)
> + list_free_deep(aggstate->hash_batches);
>
> And this is the early return mechanism per se:
>
> + if (!ExecPlanStillValid(estate))
> + return aggstate;
>
> I think at least those 3 kinds of changes deserve to be in separate
> patches with separate commit messages explaining the rationale behind
> each e.g. "Remove unnecessary cleanup calls in ExecEnd* functions.
> These calls are no longer required, because <reasons>. Removing them
> saves a few CPU cycles and simplifies planned refactoring, so do
> that."
Breaking up the patch as you describe makes sense, so I've done that:
Attached 0001 removes unnecessary cleanup calls from ExecEnd*() routines.
0002 adds NULLness checks in ExecEnd*() routines on some pointers that
may not be initialized by the corresponding ExecInit*() routines in
the case where it returns early.
0003 adds the early return mechanism based on checking CachedPlan
invalidation, though no CachedPlan is actually passed to the executor
yet, so no functional changes here yet.
Other patches are rebased over these. One significant change is in
0004 which does the refactoring to make the callers of ExecutorStart()
aware that it may now return with a partially initialized planstate
tree that should not be executed. I added a new flag
EState.es_canceled to denote that state of the execution to complement
the existing es_finished. I also needed to add
AfterTriggerCancelQuery() to ensure that we don't attempt to fire a
canceled query's triggers. Most of these changes are needed only to
appease the various Asserts in these parts of the code and I thought
they are warranted given the introduction of a new state of query
execution.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v46-0004-Make-ExecutorStart-return-early-upon-plan-invali.patch (50.0K, 2-v46-0004-Make-ExecutorStart-return-early-upon-plan-invali.patch)
download | inline diff:
From 76a2848e8f70ccbbf9c1844c5f3c49fa728ae169 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 3 Aug 2023 12:34:31 +0900
Subject: [PATCH v46 4/8] Make ExecutorStart() return early upon plan
invalidation
When passing a plan tree from a CachedPlan to the executor,
ExecutorStart() can now return a planstate tree that isn't completely
set up. This scenario occurs if the CachedPlan becomes invalidated while
it's being initialized with ExecInitNode(). Execution must be retried
with a new CachedPlan when that scenario occurs. Partially initilized
EState must be cleaned up by calling ExecutorEnd() and
FreeExecutorState().
ExecutorStart() and ExecutorStart_hook() now return a Boolean telling
the caller if the plan initialization failed.
For the replan loop in that context, it makes more sense to have
ExecutorStart() either in the same scope or closer to where
GetCachedPlan() is invoked. So this commit modifies the following
sites:
* The ExecutorStart() call in ExplainOnePlan() is moved into a new
function ExplainQueryDesc() along with CreateQueryDesc(). Callers
of ExplainOnePlan() should now call the new function first.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and a new memory
context for those. One unintended consequence is that
CommandCounterIncrement() between queries in PORTAL_MULTI_QUERY
cases is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still works because the Snapshot registered
in QueryDesc/EState is updated to account for the CCI().
This commit also adds a new flag to EState called es_canceled that
complements es_finished to denote the new scenario where
ExecutorStart() returns with a partially setup planstate tree. Also,
to reset the AFTER trigger state that would have been set up in the
ExecutorStart(), this adds a new function AfterTriggerCancelQuery()
which is called from ExecutorEnd() (not ExecutorFinish()) when
es_canceled is true.
Note that this commit by itself doesn't make any functional change,
because the CachedPlan is not passed into the executor yet.
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 8 +-
src/backend/commands/explain.c | 142 ++++---
src/backend/commands/extension.c | 3 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/portalcmds.c | 5 +-
src/backend/commands/prepare.c | 31 +-
src/backend/commands/trigger.c | 13 +
src/backend/executor/execMain.c | 57 ++-
src/backend/executor/execParallel.c | 3 +-
src/backend/executor/execUtils.c | 1 +
src/backend/executor/functions.c | 4 +-
src/backend/executor/spi.c | 48 ++-
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 345 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/commands/trigger.h | 1 +
src/include/executor/executor.h | 6 +-
src/include/nodes/execnodes.h | 3 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/portal.h | 2 +
24 files changed, 460 insertions(+), 284 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 06b65aeef5..5354dff7d7 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -324,7 +324,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -961,13 +961,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -990,6 +992,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index eaa3172793..a45489f8f5 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -567,8 +567,10 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..167db4cf56 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -329,8 +329,12 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..fe9314bc96 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,87 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +600,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +618,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4894,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..b702a65e81 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -801,7 +801,8 @@ execute_sql_string(const char *sql)
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..7124994a43 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -412,8 +412,12 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..5120f93414 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,10 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..699df429c4 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if the cached plan was found to have been invalidated
+ * when initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +647,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 52177759ab..dd139432b9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -5009,6 +5009,19 @@ AfterTriggerBeginQuery(void)
afterTriggers.query_depth++;
}
+/* ----------
+ * AfterTriggerCancelQuery()
+ *
+ * Called from ExecutorEnd() if the query execution was canceled.
+ * ----------
+ */
+void
+AfterTriggerCancelQuery(void)
+{
+ /* Set to a value denoting that no query is active. */
+ afterTriggers.query_depth = -1;
+}
+
/* ----------
* AfterTriggerEndQuery()
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index f3054cbe7e..88ebfb218b 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,14 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
+
+ /* Mark execution as canceled if plan won't be executed. */
+ estate->es_canceled = !plan_valid;
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -325,6 +338,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
estate = queryDesc->estate;
Assert(estate != NULL);
+ Assert(!estate->es_canceled);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -429,7 +443,7 @@ standard_ExecutorFinish(QueryDesc *queryDesc)
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/* This should be run once and only once per Executor instance */
- Assert(!estate->es_finished);
+ Assert(!estate->es_finished && !estate->es_canceled);
/* Switch into per-query memory context */
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
@@ -488,11 +502,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
Assert(estate != NULL);
/*
- * Check that ExecutorFinish was called, unless in EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
- Assert(estate->es_finished ||
+ Assert(estate->es_finished || estate->es_canceled ||
(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -506,6 +520,14 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
UnregisterSnapshot(estate->es_snapshot);
UnregisterSnapshot(estate->es_crosscheck_snapshot);
+ /*
+ * Cancel trigger execution too if the query execution was canceled.
+ */
+ if (estate->es_canceled &&
+ !(estate->es_top_eflags &
+ (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
+ AfterTriggerCancelQuery();
+
/*
* Must switch out of context before destroying it
*/
@@ -829,9 +851,12 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -1014,9 +1039,15 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
}
+ queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
+ queryDesc->planstate = planstate;
+ return true;
+
plan_init_suspended:
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
+ return false;
}
/*
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..f84a3a17d5 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1430,7 +1430,8 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c3f7279b06..da8a1511ac 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -151,6 +151,7 @@ CreateExecutorState(void)
estate->es_top_eflags = 0;
estate->es_instrument = 0;
estate->es_finished = false;
+ estate->es_canceled = false;
estate->es_exprcontexts = NIL;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..8cf0b3132d 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -862,7 +862,9 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..6a96d7fc22 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2674,8 +2686,23 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2877,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2923,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e4756f8be2..204002cff2 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..9a96b77f1e 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -116,86 +111,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -426,19 +341,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -489,8 +406,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
portal->sourceText,
@@ -501,30 +418,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +474,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +496,81 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,19 +583,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1193,7 +1183,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1204,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1224,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1252,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1339,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..37554727ee 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 430e3ca7dd..d4f7c29301 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -257,6 +257,7 @@ extern void ExecASTruncateTriggers(EState *estate,
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
+extern void AfterTriggerCancelQuery(void);
extern void AfterTriggerEndQuery(EState *estate);
extern void AfterTriggerFireDeferred(void);
extern void AfterTriggerEndXact(bool isCommit);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 72cbf120c5..10c5cda169 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b2a576b76d..0922be6678 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -670,6 +670,9 @@ typedef struct EState
int es_top_eflags; /* eflags passed to ExecutorStart */
int es_instrument; /* OR of InstrumentOption flags */
bool es_finished; /* true when ExecutorFinish is done */
+ bool es_canceled; /* true when execution was canceled
+ * upon encountering that plan was invalided
+ * during ExecInitNode() */
List *es_exprcontexts; /* List of ExprContexts within EState */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v46-0006-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 3-v46-0006-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From 7f6ec474c66c75124c48c62a7fc5d68d3750cc37 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v46 6/8] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 7a1dfb6364..cf269f8c53 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3305,10 +3305,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fef4c714b8..d875e11192 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v46-0005-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 4-v46-0005-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 766003a0342fb2eb659c5c8280cead5a74053c22 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v46 5/8] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..d1f4f606bf 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1229,6 +1230,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1370,15 +1372,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1399,7 +1409,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1445,6 +1456,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1534,15 +1546,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1554,7 +1574,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v46-0008-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 5-v46-0008-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 5da59305b0000098cabf508f7c0e4a4a74a0c11a Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v46 8/8] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 09a104f0a3..6a010b74df 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1650,12 +1650,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 94c8e5e875..3d1d467807 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -812,6 +812,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 0922be6678..fba1527792 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v46-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch (49.8K, 6-v46-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 8c561798798243d972ae50b3c46712c4c077876c Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v46 7/8] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/commands/copyto.c | 3 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 8 +-
src/backend/commands/extension.c | 1 +
src/backend/commands/matview.c | 2 +-
src/backend/commands/prepare.c | 2 +-
src/backend/executor/README | 39 ++++-
src/backend/executor/execMain.c | 20 ++-
src/backend/executor/execParallel.c | 9 +-
src/backend/executor/execPartition.c | 10 ++
src/backend/executor/execUtils.c | 61 +++++--
src/backend/executor/functions.c | 1 +
src/backend/executor/nodeAppend.c | 19 +++
src/backend/executor/nodeMergeAppend.c | 19 +++
src/backend/executor/spi.c | 1 +
src/backend/storage/lmgr/lmgr.c | 45 +++++
src/backend/tcop/pquery.c | 7 +-
src/backend/utils/cache/lsyscache.c | 21 +++
src/backend/utils/cache/plancache.c | 157 +++++++-----------
src/include/commands/explain.h | 2 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 1 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 156 +++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
28 files changed, 592 insertions(+), 131 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index a45489f8f5..ab8bf0df72 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 167db4cf56..e5cce4c07c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index fe9314bc96..6171a20fe2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -416,7 +416,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
- queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
params, queryEnv);
Assert(queryDesc);
@@ -429,9 +429,11 @@ ExplainOneQuery(Query *query, int cursorOptions,
/*
* ExplainQueryDesc
* Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
*/
QueryDesc *
-ExplainQueryDesc(PlannedStmt *stmt,
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv)
{
@@ -467,7 +469,7 @@ ExplainQueryDesc(PlannedStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(stmt, queryString,
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b702a65e81..93a683e312 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,6 +797,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 7124994a43..38795ce7ca 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,7 +408,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 699df429c4..156c3c5fee 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -650,7 +650,7 @@ replan:
{
QueryDesc *queryDesc;
- queryDesc = ExplainQueryDesc(pstmt, queryString,
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
into, es, paramLI, queryEnv);
if (queryDesc == NULL)
{
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..0a7bb42ccb 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,37 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to execution-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree after execution-initialization-pruning has been
+performed. It does that by looking at CachedPlan.is_valid of the CachedPlan
+passed to it. If the plan tree is indeed stale (is_valid=false), the executor
+must give up continuing to initialize it any further and return to the caller
+letting it know that the execution must be retried with a new plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +347,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 88ebfb218b..09a104f0a3 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -642,6 +642,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -875,12 +886,12 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
- estate->es_cachedplan = NULL;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -1465,7 +1476,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2897,7 +2908,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index f84a3a17d5..209f618a07 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Set up a QueryDesc for the query. While the leader might've sourced
+ * the plan tree from a CachedPlan, we don't have one here. This isn't
+ * an issue since the leader ensured the required locks, making our
+ * plan tree valid. Even as we get our own lock copies in
+ * ExecGetRangeTableRelation(), they're all already held by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index e88455368c..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index da8a1511ac..94c8e5e875 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -779,7 +779,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -792,15 +810,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -808,6 +817,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 8cf0b3132d..4ddf4fd7a9 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 588f5388c7..20330c5c58 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,25 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index c9d406c230..a8f9157192 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,25 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 6a96d7fc22..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2680,6 +2680,7 @@ replan:
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 9a96b77f1e..48cd6f4304 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -60,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -72,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -410,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -440,6 +443,7 @@ PortalStart(Portal portal, ParamListInfo params,
*/
if (!ExecutorStart(queryDesc, myeflags))
{
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
PopActiveSnapshot();
@@ -538,7 +542,7 @@ PortalStart(Portal portal, ParamListInfo params,
* Create the QueryDesc. DestReceiver will be set in
* PortalRunMulti() before calling ExecutorRun().
*/
- queryDesc = CreateQueryDesc(plan,
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -562,6 +566,7 @@ PortalStart(Portal portal, ParamListInfo params,
if (!ExecutorStart(queryDesc, myeflags))
{
PopActiveSnapshot();
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
plan_valid = false;
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 7d4168f82f..39fb0878fe 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -104,13 +104,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -792,8 +792,15 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan contains any child relations that would have been added by the
+ * planner, they would not have been locked yet, because AcquirePlannerLocks()
+ * only locks relations that would be present in the original query's range
+ * table (that is, before entering the planner). So, the plan could go stale
+ * before it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables during the plan
+ * initialization phase, and if it is not, ask the caller to recreate the
+ * plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -807,60 +814,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1129,9 +1132,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* This function hides the logic that decides whether to use a generic
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
- *
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+
+ * Upon return, the plan is generally valid. However, if it includes
+ * inheritance/partition child tables, they will not have been locked, since
+ * only tables mentioned in the original query are locked here. The executor
+ * locks these child tables when setting up the plan tree. If the plan is
+ * invalidated due to these locks, the executor should prompt the calling
+ * module to fetch a new plan by calling this function again. We defer child
+ * table locking to the executor like this because not all might need locking;
+ * some might be pruned during executor plan initialization, especially if
+ * the plan nodes under which they are scanned support partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1166,7 +1176,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1364,8 +1377,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1741,58 +1754,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 37554727ee..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,7 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv);
extern void ExplainOnePlan(QueryDesc *queryDesc,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 10c5cda169..eaa605e513 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -599,6 +599,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v46-0003-Support-for-ExecInitNode-to-detect-CachedPlan-in.patch (36.7K, 7-v46-0003-Support-for-ExecInitNode-to-detect-CachedPlan-in.patch)
download | inline diff:
From c6234c690231d0aa9cc211309e7059d5c366d06e Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 11 Aug 2023 14:09:29 +0900
Subject: [PATCH v46 3/8] Support for ExecInitNode() to detect CachedPlan
invalidation
This commit adds checks to determine if a CachedPlan remains valid
during ExecInitNode() traversal of the plan from the CachedPlan. This
includes points right after opening/locking tables and during
recursive ExecInitNode() calls to initialize child plans. Depending
on the situation, specific ExecInit*() routines will:
* Return NULL if invalidation is spotted right after opening a table
or after a function that opens one, but before initializing child
nodes.
* Return the partially initialized PlanState node if invalidation is
found after recursively initializing a child node via
ExecInitNode().
A prior commit already fortified ExecEnd*() to manage these partial
nodes, containing partially initialized nodes and missing child node
links.
Importantly, this commit doesn't alter functionality. The CachedPlan
isn't fed to the executor as of now, and the executor doesn't lock
tables.
---
contrib/postgres_fdw/postgres_fdw.c | 4 ++++
src/backend/executor/execMain.c | 24 ++++++++++++++++++++--
src/backend/executor/execPartition.c | 4 ++++
src/backend/executor/execProcnode.c | 17 ++++++++++++++-
src/backend/executor/execUtils.c | 2 ++
src/backend/executor/nodeAgg.c | 2 ++
src/backend/executor/nodeAppend.c | 14 ++++++++++---
src/backend/executor/nodeBitmapAnd.c | 11 +++++++---
src/backend/executor/nodeBitmapHeapscan.c | 4 ++++
src/backend/executor/nodeBitmapIndexscan.c | 2 ++
src/backend/executor/nodeBitmapOr.c | 11 +++++++---
src/backend/executor/nodeCustom.c | 2 ++
src/backend/executor/nodeForeignscan.c | 4 ++++
src/backend/executor/nodeGather.c | 3 +++
src/backend/executor/nodeGatherMerge.c | 2 ++
src/backend/executor/nodeGroup.c | 2 ++
src/backend/executor/nodeHash.c | 2 ++
src/backend/executor/nodeHashjoin.c | 4 ++++
src/backend/executor/nodeIncrementalSort.c | 2 ++
src/backend/executor/nodeIndexonlyscan.c | 4 ++++
src/backend/executor/nodeIndexscan.c | 4 ++++
src/backend/executor/nodeLimit.c | 2 ++
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 2 ++
src/backend/executor/nodeMemoize.c | 2 ++
src/backend/executor/nodeMergeAppend.c | 10 ++++++++-
src/backend/executor/nodeMergejoin.c | 4 ++++
src/backend/executor/nodeModifyTable.c | 7 +++++++
src/backend/executor/nodeNestloop.c | 4 ++++
src/backend/executor/nodeProjectSet.c | 2 ++
src/backend/executor/nodeRecursiveunion.c | 4 ++++
src/backend/executor/nodeResult.c | 2 ++
src/backend/executor/nodeSamplescan.c | 2 ++
src/backend/executor/nodeSeqscan.c | 2 ++
src/backend/executor/nodeSetOp.c | 2 ++
src/backend/executor/nodeSort.c | 2 ++
src/backend/executor/nodeSubqueryscan.c | 2 ++
src/backend/executor/nodeTidrangescan.c | 2 ++
src/backend/executor/nodeTidscan.c | 2 ++
src/backend/executor/nodeUnique.c | 2 ++
src/backend/executor/nodeWindowAgg.c | 2 ++
src/include/executor/executor.h | 10 +++++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 +++++++++++++
44 files changed, 196 insertions(+), 13 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1393716587..ab7ecb925c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2660,7 +2660,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..f3054cbe7e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -839,8 +839,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
- TupleDesc tupType;
+ PlanState *planstate = NULL;
+ TupleDesc tupType = NULL;
ListCell *l;
int i;
@@ -855,6 +855,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = NULL;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +887,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -956,6 +959,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
i++;
}
@@ -966,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1007,6 +1014,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
}
+plan_init_suspended:
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
}
@@ -2945,6 +2953,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2988,6 +3002,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..e88455368c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1801,6 +1801,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1929,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 6098cdca69..d5952d0d50 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,18 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * Various node type specific ExecInit* routines listed below either
+ * return NULL or a partially initialized PlanState tree if the CachedPlan
+ * is found to be invalidated. That is checked by calling
+ * ExecPlanStillValid() at various points, such as after opening/locking
+ * a relation, or after calling a function that does which includes
+ * recursive invocations of ExecInitNode() to initialize child nodes.
+ * A given ExecInit* routine should return NULL upon getting false from
+ * ExecPlanStillValid() if no child node has been initialzed at the point
+ * of checking and the partially initialized PlanState node if a child
+ * node has been recursively initialized.
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +399,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ return result;
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 16704c0c2f..c3f7279b06 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -822,6 +822,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index aac9e9fc80..f46c3df199 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3305,6 +3305,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return aggstate;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..588f5388c7 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -147,6 +147,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -185,8 +187,13 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ /*
+ * Any uninitialized sunbodes will have NULL in appendplans in the case of
+ * an early return.
+ */
+ appendstate->appendplans = appendplanstates =
+ (PlanState **) palloc0(nplans * sizeof(PlanState *));
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +228,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..c0495ec90f 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -69,6 +69,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
*/
nplans = list_length(node->bitmapplans);
+ /*
+ * Any uninitialized sunbodes will have NULL in bitmapplans in the case of
+ * an early return.
+ */
bitmapplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
/*
@@ -78,7 +82,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
bitmapandstate->ps.state = estate;
bitmapandstate->ps.ExecProcNode = ExecBitmapAnd;
bitmapandstate->bitmapplans = bitmapplanstates;
- bitmapandstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -88,8 +91,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmapandstate;
+ bitmapandstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ffa51c06b4..3cdece852c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -752,11 +752,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 7cf8532bc9..4200472d02 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -255,6 +255,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return indexstate;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..00120669a5 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -70,6 +70,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
*/
nplans = list_length(node->bitmapplans);
+ /*
+ * Any uninitialized sunbodes will have NULL in bitmapplans in the case of
+ * an early return.
+ */
bitmapplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
/*
@@ -79,7 +83,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
bitmaporstate->ps.state = estate;
bitmaporstate->ps.ExecProcNode = ExecBitmapOr;
bitmaporstate->bitmapplans = bitmapplanstates;
- bitmaporstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -89,8 +92,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmaporstate;
+ bitmaporstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index e80be3af81..76f5c2fd09 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index d5aaa983f7..0eeb66530a 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index bb2500a469..6b26e03f74 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gatherstate;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 7a71a58509..84412f94bb 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 8c650f0e46..b6068887f6 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return grpstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index e72f0986c2..030bf0ed43 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index aea44a9d56..49a6ba4276 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index dcb8470ba7..6caa1aa306 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return incrsortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f1db35665c..ea7fd89c0c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -496,6 +496,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -549,6 +551,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 14b9c00217..906358011a 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -909,6 +909,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -954,6 +956,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 5654158e3e..6760de0f25 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return limitstate;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..2599332f01 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 753ea28915..b974ebdc8a 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 5352ca10c8..d0cdbe1fd7 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..c9d406c230 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -95,6 +95,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -120,7 +122,11 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ /*
+ * Any uninitialized sunbodes will have NULL in mergeplans in the case of
+ * an early return.
+ */
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +157,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 648fdd9a5f..e7f4512419 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d21a178ad5..c28d5058e9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3985,6 +3985,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4012,6 +4015,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4039,6 +4044,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index fc8f833d8b..0158a3e592 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index b4bbdc89b1..1b4774d4f7 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return state;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index 3dfcb4cafb..ca4f78685d 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index e9f5732f33..d4ea101cbe 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return resstate;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1aa0e2a205..edda889e55 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 49a5933aff..48e20aa735 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 98c1b84d43..7a3a142204 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index eea7f2ae15..3ebbc46604 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return sortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 1ee6295660..3c5c7c2ebb 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index da622d3f5f..d337f3d54a 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -374,6 +374,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 15055077d0..9637f354b2 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -517,6 +517,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 01f951197c..28630e380e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 3849d2f847..04d4eebce4 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2461,6 +2461,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index aeebe0e0ff..72cbf120c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,15 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the CachedPlan in es_cachedplan still valid?
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..b2a576b76d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..c83a67fea3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
[application/octet-stream] v46-0001-Refactor-ExecEnd-routines-to-enhance-efficiency.patch (30.5K, 8-v46-0001-Refactor-ExecEnd-routines-to-enhance-efficiency.patch)
download | inline diff:
From ec8faad9bc9ae157ebca85a7892857a04f06fb39 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 1 Sep 2023 17:46:32 +0900
Subject: [PATCH v46 1/8] Refactor ExecEnd* routines to enhance efficiency
This commit removes unnecessary ExecExprFreeContext() calls in ExecEnd*
routines as the actual cleanup is managed by FreeExecutorState. With
no remaining callers for ExecExprFreeContext(), this commit also
removes the function.
This commit also drops redundant ExecClearTuple() calls, as
ExecResetTupleTable() in ExecEndPlan() already takes care of resetting
all TupleTableSlots.
After these modifications, the ExecEnd*() routines for ValuesScan,
NamedTuplestoreScan, and WorkTableScan became redundant. Thus, this
commit removes them. These changes not only optimize CPU usage during
ExecEndNode() processing but also pave the way for an upcoming patch.
This future patch aims to allow ExecEndNode() to expect PlanState
trees that are only partially initialized in some cases.
---
src/backend/executor/execProcnode.c | 18 +++++--------
src/backend/executor/execUtils.c | 26 -------------------
src/backend/executor/nodeAgg.c | 10 -------
src/backend/executor/nodeBitmapHeapscan.c | 12 ---------
src/backend/executor/nodeBitmapIndexscan.c | 8 ------
src/backend/executor/nodeCtescan.c | 13 +---------
src/backend/executor/nodeCustom.c | 8 +-----
src/backend/executor/nodeFunctionscan.c | 12 ---------
src/backend/executor/nodeGather.c | 3 ---
src/backend/executor/nodeGatherMerge.c | 3 ---
src/backend/executor/nodeGroup.c | 5 ----
src/backend/executor/nodeHash.c | 5 ----
src/backend/executor/nodeHashjoin.c | 12 ---------
src/backend/executor/nodeIncrementalSort.c | 8 ------
src/backend/executor/nodeIndexonlyscan.c | 16 ------------
src/backend/executor/nodeIndexscan.c | 16 ------------
src/backend/executor/nodeLimit.c | 1 -
src/backend/executor/nodeMaterial.c | 5 ----
src/backend/executor/nodeMemoize.c | 9 -------
src/backend/executor/nodeMergejoin.c | 12 ---------
src/backend/executor/nodeModifyTable.c | 11 --------
.../executor/nodeNamedtuplestorescan.c | 22 ----------------
src/backend/executor/nodeNestloop.c | 11 --------
src/backend/executor/nodeProjectSet.c | 10 -------
src/backend/executor/nodeResult.c | 10 -------
src/backend/executor/nodeSamplescan.c | 13 +---------
src/backend/executor/nodeSeqscan.c | 12 ---------
src/backend/executor/nodeSetOp.c | 4 ---
src/backend/executor/nodeSort.c | 7 -----
src/backend/executor/nodeSubqueryscan.c | 12 ---------
src/backend/executor/nodeTableFuncscan.c | 12 ---------
src/backend/executor/nodeTidrangescan.c | 12 ---------
src/backend/executor/nodeTidscan.c | 12 ---------
src/backend/executor/nodeUnique.c | 5 ----
src/backend/executor/nodeValuesscan.c | 24 -----------------
src/backend/executor/nodeWindowAgg.c | 17 ------------
src/backend/executor/nodeWorktablescan.c | 22 ----------------
src/include/executor/executor.h | 1 -
.../executor/nodeNamedtuplestorescan.h | 1 -
src/include/executor/nodeValuesscan.h | 1 -
src/include/executor/nodeWorktablescan.h | 1 -
41 files changed, 9 insertions(+), 413 deletions(-)
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..6098cdca69 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -667,22 +667,10 @@ ExecEndNode(PlanState *node)
ExecEndTableFuncScan((TableFuncScanState *) node);
break;
- case T_ValuesScanState:
- ExecEndValuesScan((ValuesScanState *) node);
- break;
-
case T_CteScanState:
ExecEndCteScan((CteScanState *) node);
break;
- case T_NamedTuplestoreScanState:
- ExecEndNamedTuplestoreScan((NamedTuplestoreScanState *) node);
- break;
-
- case T_WorkTableScanState:
- ExecEndWorkTableScan((WorkTableScanState *) node);
- break;
-
case T_ForeignScanState:
ExecEndForeignScan((ForeignScanState *) node);
break;
@@ -757,6 +745,12 @@ ExecEndNode(PlanState *node)
ExecEndLimit((LimitState *) node);
break;
+ /* No clean up actions for these nodes. */
+ case T_ValuesScanState:
+ case T_NamedTuplestoreScanState:
+ case T_WorkTableScanState:
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(node));
break;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..16704c0c2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -638,32 +638,6 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, int varno, TupleDesc tupdesc)
return true;
}
-/* ----------------
- * ExecFreeExprContext
- *
- * A plan node's ExprContext should be freed explicitly during executor
- * shutdown because there may be shutdown callbacks to call. (Other resources
- * made by the above routines, such as projection info, don't need to be freed
- * explicitly because they're just memory in the per-query memory context.)
- *
- * However ... there is no particular need to do it during ExecEndNode,
- * because FreeExecutorState will free any remaining ExprContexts within
- * the EState. Letting FreeExecutorState do it allows the ExprContexts to
- * be freed in reverse order of creation, rather than order of creation as
- * will happen if we delete them here, which saves O(N^2) work in the list
- * cleanup inside FreeExprContext.
- * ----------------
- */
-void
-ExecFreeExprContext(PlanState *planstate)
-{
- /*
- * Per above discussion, don't actually delete the ExprContext. We do
- * unlink it from the plan node, though.
- */
- planstate->ps_ExprContext = NULL;
-}
-
/* ----------------------------------------------------------------
* Scan node support
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..f154f28902 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4357,16 +4357,6 @@ ExecEndAgg(AggState *node)
if (node->hashcontext)
ReScanExprContext(node->hashcontext);
- /*
- * We don't actually free any ExprContexts here (see comment in
- * ExecFreeExprContext), just unlinking the output one from the plan node
- * suffices.
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..2db0acfc76 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -655,18 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..7cf8532bc9 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -184,14 +184,6 @@ ExecEndBitmapIndexScan(BitmapIndexScanState *node)
indexRelationDesc = node->biss_RelationDesc;
indexScanDesc = node->biss_ScanDesc;
- /*
- * Free the exprcontext ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- if (node->biss_RuntimeContext)
- FreeExprContext(node->biss_RuntimeContext, true);
-#endif
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index cc4c4243e2..14e010c0ea 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -287,23 +287,12 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
void
ExecEndCteScan(CteScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* If I am the leader, free the tuplestore.
*/
if (node->leader == node)
{
+ Assert(node->cte_table);
tuplestore_end(node->cte_table);
node->cte_table = NULL;
}
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..e80be3af81 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -127,15 +127,9 @@ ExecCustomScan(PlanState *pstate)
void
ExecEndCustomScan(CustomScanState *node)
{
+ Assert(node->methods);
Assert(node->methods->EndCustomScan != NULL);
node->methods->EndCustomScan(node);
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* Clean out the tuple table */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
void
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index dd06ef8aee..a49c1a2c85 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -523,18 +523,6 @@ ExecEndFunctionScan(FunctionScanState *node)
{
int i;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release slots and tuplestore resources
*/
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..bb2500a469 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -250,9 +250,6 @@ ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..7a71a58509 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -290,9 +290,6 @@ ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..8c650f0e46 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -228,11 +228,6 @@ ExecEndGroup(GroupState *node)
{
PlanState *outerPlan;
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..e72f0986c2 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -415,11 +415,6 @@ ExecEndHash(HashState *node)
{
PlanState *outerPlan;
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..aea44a9d56 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -867,18 +867,6 @@ ExecEndHashJoin(HashJoinState *node)
node->hj_HashTable = NULL;
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->hj_OuterTupleSlot);
- ExecClearTuple(node->hj_HashTupleSlot);
-
/*
* clean up subtrees
*/
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..dcb8470ba7 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,14 +1079,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- /* clean out the scan tuple */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- /* must drop standalone tuple slots from outer node */
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
-
/*
* Release tuplesort resources.
*/
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..f1db35665c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -380,22 +380,6 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->ioss_RuntimeContext)
- FreeExprContext(node->ioss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..14b9c00217 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -794,22 +794,6 @@ ExecEndIndexScan(IndexScanState *node)
indexRelationDesc = node->iss_RelationDesc;
indexScanDesc = node->iss_ScanDesc;
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->iss_RuntimeContext)
- FreeExprContext(node->iss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..5654158e3e 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -534,7 +534,6 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
void
ExecEndLimit(LimitState *node)
{
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..753ea28915 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -239,11 +239,6 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
void
ExecEndMaterial(MaterialState *node)
{
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..94bf479287 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1091,15 +1091,6 @@ ExecEndMemoize(MemoizeState *node)
/* Remove the cache context */
MemoryContextDelete(node->tableContext);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to cache result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..648fdd9a5f 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1642,18 +1642,6 @@ ExecEndMergeJoin(MergeJoinState *node)
{
MJ1_printf("ExecEndMergeJoin: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->mj_MarkedTupleSlot);
-
/*
* shut down the subplans
*/
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5005d8c0d1..d21a178ad5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4446,17 +4446,6 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* Terminate EPQ execution if active
*/
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index 46832ad82f..3547dc2b10 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -155,28 +155,6 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndNamedTuplestoreScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanNamedTuplestoreScan
*
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..fc8f833d8b 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -363,17 +363,6 @@ ExecEndNestLoop(NestLoopState *node)
{
NL1_printf("ExecEndNestLoop: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..b4bbdc89b1 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -320,16 +320,6 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
void
ExecEndProjectSet(ProjectSetState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..e9f5732f33 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -240,16 +240,6 @@ ExecInitResult(Result *node, EState *estate, int eflags)
void
ExecEndResult(ResultState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..1aa0e2a205 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -185,21 +185,10 @@ ExecEndSampleScan(SampleScanState *node)
/*
* Tell sampling function that we finished the scan.
*/
+ Assert(node->tsmroutine);
if (node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..49a5933aff 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -190,18 +190,6 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..98c1b84d43 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -582,13 +582,9 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
void
ExecEndSetOp(SetOpState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/* free subsidiary stuff including hashtable */
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..eea7f2ae15 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -303,13 +303,6 @@ ExecEndSort(SortState *node)
SO1_printf("ExecEndSort: %s\n",
"shutting down sort node");
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
/*
* Release tuplesort resources
*/
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..1ee6295660 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -167,18 +167,6 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
void
ExecEndSubqueryScan(SubqueryScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the upper tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subquery
*/
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 791cbd2372..a60dcd4943 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -213,18 +213,6 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
void
ExecEndTableFuncScan(TableFuncScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..da622d3f5f 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -331,18 +331,6 @@ ExecEndTidRangeScan(TidRangeScanState *node)
if (scan != NULL)
table_endscan(scan);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..15055077d0 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -472,18 +472,6 @@ ExecEndTidScan(TidScanState *node)
{
if (node->ss.ss_currentScanDesc)
table_endscan(node->ss.ss_currentScanDesc);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..01f951197c 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -168,11 +168,6 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
void
ExecEndUnique(UniqueState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- ExecFreeExprContext(&node->ps);
-
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index 32ace63017..fbfb067f3b 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -319,30 +319,6 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndValuesScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndValuesScan(ValuesScanState *node)
-{
- /*
- * Free both exprcontexts
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->rowcontext;
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanValuesScan
*
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..77724a6daa 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2686,23 +2686,6 @@ ExecEndWindowAgg(WindowAggState *node)
release_partition(node);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- ExecClearTuple(node->first_part_slot);
- ExecClearTuple(node->agg_row_slot);
- ExecClearTuple(node->temp_slot_1);
- ExecClearTuple(node->temp_slot_2);
- if (node->framehead_slot)
- ExecClearTuple(node->framehead_slot);
- if (node->frametail_slot)
- ExecClearTuple(node->frametail_slot);
-
- /*
- * Free both the expr contexts.
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->tmpcontext;
- ExecFreeExprContext(&node->ss.ps);
-
for (i = 0; i < node->numaggs; i++)
{
if (node->peragg[i].aggcontext != node->aggcontext)
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 0c13448236..17a548865e 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -181,28 +181,6 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndWorkTableScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndWorkTableScan(WorkTableScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanWorkTableScan
*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..aeebe0e0ff 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -569,7 +569,6 @@ extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
-extern void ExecFreeExprContext(PlanState *planstate);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/nodeNamedtuplestorescan.h b/src/include/executor/nodeNamedtuplestorescan.h
index 3ff687023a..9d80236fe5 100644
--- a/src/include/executor/nodeNamedtuplestorescan.h
+++ b/src/include/executor/nodeNamedtuplestorescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern NamedTuplestoreScanState *ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflags);
-extern void ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node);
extern void ExecReScanNamedTuplestoreScan(NamedTuplestoreScanState *node);
#endif /* NODENAMEDTUPLESTORESCAN_H */
diff --git a/src/include/executor/nodeValuesscan.h b/src/include/executor/nodeValuesscan.h
index a52fa678df..fe3f043951 100644
--- a/src/include/executor/nodeValuesscan.h
+++ b/src/include/executor/nodeValuesscan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern ValuesScanState *ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags);
-extern void ExecEndValuesScan(ValuesScanState *node);
extern void ExecReScanValuesScan(ValuesScanState *node);
#endif /* NODEVALUESSCAN_H */
diff --git a/src/include/executor/nodeWorktablescan.h b/src/include/executor/nodeWorktablescan.h
index e553a453f3..f31b22cec4 100644
--- a/src/include/executor/nodeWorktablescan.h
+++ b/src/include/executor/nodeWorktablescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern WorkTableScanState *ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags);
-extern void ExecEndWorkTableScan(WorkTableScanState *node);
extern void ExecReScanWorkTableScan(WorkTableScanState *node);
#endif /* NODEWORKTABLESCAN_H */
--
2.35.3
[application/octet-stream] v46-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch (5.8K, 9-v46-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch)
download | inline diff:
From 3dfe81f48a58a92e8c81469600d3502f18a8b137 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 1 Sep 2023 22:05:35 +0900
Subject: [PATCH v46 2/8] Check pointer NULLness before cleanup in ExecEnd*
routines
Many routines already perform this check, but a few instances remain.
Currently, these NULLness checks might seem redundant since ExecEnd*
routines operate under the assumption that their matching ExecInit*
routine would have fully executed, ensuring pointers are set. However,
a forthcoming patch will modify ExecInit* routines to sometimes exit
early, potentially leaving some pointers in an undetermined state.
---
src/backend/executor/nodeAgg.c | 3 ++-
src/backend/executor/nodeBitmapHeapscan.c | 3 ++-
src/backend/executor/nodeForeignscan.c | 21 ++++++++------------
src/backend/executor/nodeMemoize.c | 1 +
src/backend/executor/nodeRecursiveunion.c | 6 ++++--
src/backend/executor/nodeWindowAgg.c | 24 +++++++++++++++--------
6 files changed, 33 insertions(+), 25 deletions(-)
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index f154f28902..aac9e9fc80 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3150,7 +3150,8 @@ hashagg_reset_spill_state(AggState *aggstate)
}
/* free batches */
- list_free_deep(aggstate->hash_batches);
+ if (aggstate->hash_batches)
+ list_free_deep(aggstate->hash_batches);
aggstate->hash_batches = NIL;
/* close tape set */
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2db0acfc76..ffa51c06b4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -681,7 +681,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..d5aaa983f7 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -301,25 +301,20 @@ ExecEndForeignScan(ForeignScanState *node)
EState *estate = node->ss.ps.state;
/* Let the FDW shut down */
- if (plan->operation != CMD_SELECT)
+ if (node->fdwroutine)
{
- if (estate->es_epq_active == NULL)
- node->fdwroutine->EndDirectModify(node);
+ if (plan->operation != CMD_SELECT)
+ {
+ if (estate->es_epq_active == NULL)
+ node->fdwroutine->EndDirectModify(node);
+ }
+ else
+ node->fdwroutine->EndForeignScan(node);
}
- else
- node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
ExecEndNode(outerPlanState(node));
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean out the tuple table */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 94bf479287..5352ca10c8 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1043,6 +1043,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable)
{
int count;
uint64 mem = 0;
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..3dfcb4cafb 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -272,8 +272,10 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table)
+ tuplestore_end(node->working_table);
+ if (node->intermediate_table)
+ tuplestore_end(node->intermediate_table);
/* free subsidiary stuff including hashtable */
if (node->tempContext)
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 77724a6daa..3849d2f847 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -1351,11 +1351,14 @@ release_partition(WindowAggState *winstate)
* any aggregate temp data). We don't rely on retail pfree because some
* aggregates might have allocated data we don't have direct pointers to.
*/
- MemoryContextResetAndDeleteChildren(winstate->partcontext);
- MemoryContextResetAndDeleteChildren(winstate->aggcontext);
+ if (winstate->partcontext)
+ MemoryContextResetAndDeleteChildren(winstate->partcontext);
+ if (winstate->aggcontext)
+ MemoryContextResetAndDeleteChildren(winstate->aggcontext);
for (i = 0; i < winstate->numaggs; i++)
{
- if (winstate->peragg[i].aggcontext != winstate->aggcontext)
+ if (winstate->peragg[i].aggcontext &&
+ winstate->peragg[i].aggcontext != winstate->aggcontext)
MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext);
}
@@ -2688,14 +2691,19 @@ ExecEndWindowAgg(WindowAggState *node)
for (i = 0; i < node->numaggs; i++)
{
- if (node->peragg[i].aggcontext != node->aggcontext)
+ if (node->peragg[i].aggcontext &&
+ node->peragg[i].aggcontext != node->aggcontext)
MemoryContextDelete(node->peragg[i].aggcontext);
}
- MemoryContextDelete(node->partcontext);
- MemoryContextDelete(node->aggcontext);
+ if (node->partcontext)
+ MemoryContextDelete(node->partcontext);
+ if (node->aggcontext)
+ MemoryContextDelete(node->aggcontext);
- pfree(node->perfunc);
- pfree(node->peragg);
+ if (node->perfunc)
+ pfree(node->perfunc);
+ if (node->peragg)
+ pfree(node->peragg);
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-05 14:41 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Robert Haas @ 2023-09-05 14:41 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Tue, Sep 5, 2023 at 3:13 AM Amit Langote <[email protected]> wrote:
> Attached 0001 removes unnecessary cleanup calls from ExecEnd*() routines.
It also adds a few random Assert()s to verify that unrelated pointers
are not NULL. I suggest that it shouldn't do that.
The commit message doesn't mention the removal of the calls to
ExecDropSingleTupleTableSlot. It's not clear to me why that's OK and I
think it would be nice to mention it in the commit message, assuming
that it is in fact OK.
I suggest changing the subject line of the commit to something like
"Remove obsolete executor cleanup code."
> 0002 adds NULLness checks in ExecEnd*() routines on some pointers that
> may not be initialized by the corresponding ExecInit*() routines in
> the case where it returns early.
I think you should only add these where it's needed. For example, I
think list_free_deep(NIL) is fine.
The changes to ExecEndForeignScan look like they include stuff that
belongs in 0001.
Personally, I prefer explicit NULL-tests i.e. if (x != NULL) to
implicit ones like if (x), but opinions vary.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-06 09:12 Amit Langote <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-09-06 09:12 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Tue, Sep 5, 2023 at 11:41 PM Robert Haas <[email protected]> wrote:
> On Tue, Sep 5, 2023 at 3:13 AM Amit Langote <[email protected]> wrote:
> > Attached 0001 removes unnecessary cleanup calls from ExecEnd*() routines.
>
> It also adds a few random Assert()s to verify that unrelated pointers
> are not NULL. I suggest that it shouldn't do that.
OK, removed.
> The commit message doesn't mention the removal of the calls to
> ExecDropSingleTupleTableSlot. It's not clear to me why that's OK and I
> think it would be nice to mention it in the commit message, assuming
> that it is in fact OK.
That is not OK, so I dropped their removal. I think I confused them
with slots in other functions initialized with
ExecInitExtraTupleSlot() that *are* put into the estate.
> I suggest changing the subject line of the commit to something like
> "Remove obsolete executor cleanup code."
Sure.
> > 0002 adds NULLness checks in ExecEnd*() routines on some pointers that
> > may not be initialized by the corresponding ExecInit*() routines in
> > the case where it returns early.
>
> I think you should only add these where it's needed. For example, I
> think list_free_deep(NIL) is fine.
OK, done.
> The changes to ExecEndForeignScan look like they include stuff that
> belongs in 0001.
Oops, yes. Moved to 0001.
> Personally, I prefer explicit NULL-tests i.e. if (x != NULL) to
> implicit ones like if (x), but opinions vary.
I agree, so changed all the new tests to use (x != NULL) form.
Typically, I try to stick with whatever style is used in the nearby
code, though I can see both styles being used in the ExecEnd*()
routines. I opted to use the style that we both happen to prefer.
Attached updated patches. Thanks for the review.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v47-0005-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 2-v47-0005-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 32b706d61e4517654e85676477676cc11f682f63 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:02 +0900
Subject: [PATCH v47 5/8] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..d1f4f606bf 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1229,6 +1230,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1370,15 +1372,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1399,7 +1409,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1445,6 +1456,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1534,15 +1546,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1554,7 +1574,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v47-0006-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 3-v47-0006-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From c24beb3f1a74a9d3d485b85ea2a3026f71674aaa Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:11 +0900
Subject: [PATCH v47 6/8] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 7a1dfb6364..cf269f8c53 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3305,10 +3305,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fef4c714b8..d875e11192 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v47-0008-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 4-v47-0008-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From f28f8e58d3fd4120e4710f5aa9fe4060d064df22 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:19 +0900
Subject: [PATCH v47 8/8] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 9a3f6c5978..9d88cf30cb 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1650,12 +1650,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 94c8e5e875..3d1d467807 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -812,6 +812,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 0922be6678..fba1527792 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v47-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch (49.3K, 5-v47-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 3acf239d7d3e4ca1db1aab2b258590ccd6cda87b Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:15 +0900
Subject: [PATCH v47 7/8] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/commands/copyto.c | 3 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 8 +-
src/backend/commands/extension.c | 1 +
src/backend/commands/matview.c | 2 +-
src/backend/commands/prepare.c | 2 +-
src/backend/executor/README | 36 +++-
src/backend/executor/execMain.c | 18 +-
src/backend/executor/execParallel.c | 9 +-
src/backend/executor/execPartition.c | 10 ++
src/backend/executor/execUtils.c | 61 +++++--
src/backend/executor/functions.c | 1 +
src/backend/executor/nodeAppend.c | 19 +++
src/backend/executor/nodeMergeAppend.c | 19 +++
src/backend/executor/spi.c | 1 +
src/backend/storage/lmgr/lmgr.c | 45 +++++
src/backend/tcop/pquery.c | 7 +-
src/backend/utils/cache/lsyscache.c | 21 +++
src/backend/utils/cache/plancache.c | 154 +++++++----------
src/include/commands/explain.h | 2 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 1 +
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 156 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
28 files changed, 586 insertions(+), 129 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index a45489f8f5..ab8bf0df72 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 167db4cf56..e5cce4c07c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index fe9314bc96..6171a20fe2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -416,7 +416,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
- queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
params, queryEnv);
Assert(queryDesc);
@@ -429,9 +429,11 @@ ExplainOneQuery(Query *query, int cursorOptions,
/*
* ExplainQueryDesc
* Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
*/
QueryDesc *
-ExplainQueryDesc(PlannedStmt *stmt,
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv)
{
@@ -467,7 +469,7 @@ ExplainQueryDesc(PlannedStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(stmt, queryString,
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b702a65e81..93a683e312 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,6 +797,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 7124994a43..38795ce7ca 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,7 +408,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index bcdf56fe32..f8d0b0ee25 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -650,7 +650,7 @@ replan:
{
QueryDesc *queryDesc;
- queryDesc = ExplainQueryDesc(pstmt, queryString,
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
into, es, paramLI, queryEnv);
if (queryDesc == NULL)
{
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..6d2240610d 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,34 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Typically, when the executor initializes a plan tree for execution, it doesn't
+lock non-index relations if the plan tree is freshly generated and not derived
+from a CachedPlan. This is because such locks have already been established
+during the query's parsing, rewriting, and planning phases. However, with a
+cached plan tree, there can be relations that remain unlocked. The function
+GetCachedPlan() locks relations existing in the query's range table pre-planning
+but doesn't account for those added during the planning phase. Consequently,
+inheritance child tables, introduced to the query's range table during planning,
+won't be locked when the cached plan reaches the executor.
+
+The decision to defer locking child tables with GetCachedPlan() arises from the
+fact that not all might be accessed during plan execution. For instance, if
+child tables are partitions, some might be omitted due to pruning at
+execution-initialization-time. Thus, the responsibility of locking these child
+tables is pushed to execution-initialization-time, taking place in ExecInitNode()
+for plan nodes encompassing these tables.
+
+This approach opens a window where a cached plan tree with child tables could
+become outdated if another backend modifies these tables before ExecInitNode()
+locks them. Given this, the executor has the added duty to confirm the plan
+tree's validity whenever it locks a child table post execution-initialization-
+pruning. This validation is done by checking the CachedPlan.is_valid attribute
+of the CachedPlan provided. If the plan tree is outdated (is_valid=false), the
+executor halts any further initialization and alerts the caller that they should
+retry execution with another freshly created plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +344,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 383ebee008..9a3f6c5978 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -642,6 +642,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -880,7 +891,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
- estate->es_cachedplan = NULL;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -1465,7 +1476,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2897,7 +2908,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index f84a3a17d5..209f618a07 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Set up a QueryDesc for the query. While the leader might've sourced
+ * the plan tree from a CachedPlan, we don't have one here. This isn't
+ * an issue since the leader ensured the required locks, making our
+ * plan tree valid. Even as we get our own lock copies in
+ * ExecGetRangeTableRelation(), they're all already held by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index e88455368c..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index da8a1511ac..94c8e5e875 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -779,7 +779,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -792,15 +810,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -808,6 +817,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 8cf0b3132d..4ddf4fd7a9 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 588f5388c7..20330c5c58 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,25 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index c9d406c230..a8f9157192 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,25 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 6a96d7fc22..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2680,6 +2680,7 @@ replan:
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 9a96b77f1e..48cd6f4304 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -60,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -72,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -410,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -440,6 +443,7 @@ PortalStart(Portal portal, ParamListInfo params,
*/
if (!ExecutorStart(queryDesc, myeflags))
{
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
PopActiveSnapshot();
@@ -538,7 +542,7 @@ PortalStart(Portal portal, ParamListInfo params,
* Create the QueryDesc. DestReceiver will be set in
* PortalRunMulti() before calling ExecutorRun().
*/
- queryDesc = CreateQueryDesc(plan,
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -562,6 +566,7 @@ PortalStart(Portal portal, ParamListInfo params,
if (!ExecutorStart(queryDesc, myeflags))
{
PopActiveSnapshot();
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
plan_valid = false;
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 7d4168f82f..35d903cb98 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -104,13 +104,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -792,8 +792,13 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan includes child relations introduced by the planner, they
+ * wouldn't be locked yet. This is because AcquirePlannerLocks() only locks
+ * relations present in the original query's range table (before planner
+ * entry). Hence, the plan might become stale if child relations are modified
+ * concurrently. During the plan initialization, the executor must ensure the
+ * plan (CachedPlan) remains valid after locking each child table. If found
+ * invalid, the caller should be prompted to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -807,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1130,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * Typically, the plan returned by this function is valid. However, a caveat
+ * arises with inheritance/partition child tables. These aren't locked by
+ * this function, as we only lock tables directly mentioned in the original
+ * query here. The task of locking these child tables falls to the executor
+ * during plan tree setup. If acquiring these locks invalidates the plan, the
+ * executor should inform the caller to regenerate the plan by invoking this
+ * function again. The reason for this deferred child table locking mechanism
+ * is efficiency: not all might need to be locked. Some could be pruned during
+ * executor initialization, especially if their corresponding plan nodes
+ * facilitate partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1166,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1364,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1741,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 37554727ee..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,7 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv);
extern void ExplainOnePlan(QueryDesc *queryDesc,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 10c5cda169..eaa605e513 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -599,6 +599,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v47-0004-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch (50.0K, 6-v47-0004-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch)
download | inline diff:
From f9ff9aa3ddc05ee948d721a05ec552d5e959c498 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:46 +0900
Subject: [PATCH v47 4/8] Adjustments to allow ExecutorStart() to sometimes
fail
Upon passing a plan tree from a CachedPlan to the executor, there's a
possibility that ExecutorStart() might return an incompletely set up
planstate tree. This can happen if the CachedPlan undergoes invalidation
during the ExecInitNode() initialization process. In such cases, the
execution should be reattempted using a fresh CachedPlan. Also, any
partially initialized EState must be cleaned up by invoking both
ExecutorEnd() and FreeExecutorState().
ExecutorStart() (and ExecutorStart_hook()) now return a Boolean telling
the caller if the plan initialization failed.
For the replan loop in that context, it makes more sense to have
ExecutorStart() either in the same scope or closer to where
GetCachedPlan() is invoked. So this commit modifies the following
sites:
* The ExecutorStart() call in ExplainOnePlan() is moved into a new
function ExplainQueryDesc() along with CreateQueryDesc(). Callers
of ExplainOnePlan() should now call the new function first.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and a new memory
context for those. One unintended consequence is that
CommandCounterIncrement() between queries in the PORTAL_MULTI_QUERY
case is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still works because the Snapshot registered
in QueryDesc/EState is updated to account for the CCI().
This commit also adds a new flag to EState called es_canceled that
complements es_finished to denote the new scenario where
ExecutorStart() returns with a partially setup planstate tree. Also,
to reset the AFTER trigger state that would have been set up in the
ExecutorStart(), this adds a new function AfterTriggerCancelQuery()
which is called from ExecutorEnd() (not ExecutorFinish()) when
es_canceled is true.
Note that this commit by itself doesn't make any functional change,
because the CachedPlan is not passed into the executor yet.
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 4 +-
src/backend/commands/createas.c | 8 +-
src/backend/commands/explain.c | 142 ++++---
src/backend/commands/extension.c | 3 +-
src/backend/commands/matview.c | 8 +-
src/backend/commands/portalcmds.c | 5 +-
src/backend/commands/prepare.c | 31 +-
src/backend/commands/trigger.c | 13 +
src/backend/executor/execMain.c | 57 ++-
src/backend/executor/execParallel.c | 3 +-
src/backend/executor/execUtils.c | 1 +
src/backend/executor/functions.c | 4 +-
src/backend/executor/spi.c | 48 ++-
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 345 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/commands/trigger.h | 1 +
src/include/executor/executor.h | 6 +-
src/include/nodes/execnodes.h | 3 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/portal.h | 2 +
24 files changed, 460 insertions(+), 284 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 06b65aeef5..5354dff7d7 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -324,7 +324,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -961,13 +961,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -990,6 +992,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index eaa3172793..a45489f8f5 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -567,8 +567,10 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..167db4cf56 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -329,8 +329,12 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..fe9314bc96 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,87 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +600,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +618,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4865,6 +4894,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..b702a65e81 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -801,7 +801,8 @@ execute_sql_string(const char *sql)
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..7124994a43 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -412,8 +412,12 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..5120f93414 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,10 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..bcdf56fe32 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal has a cached plan and
+ * it's found to be invalidated during the initialization of its plan
+ * trees, the plan must be regenerated.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +647,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 52177759ab..dd139432b9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -5009,6 +5009,19 @@ AfterTriggerBeginQuery(void)
afterTriggers.query_depth++;
}
+/* ----------
+ * AfterTriggerCancelQuery()
+ *
+ * Called from ExecutorEnd() if the query execution was canceled.
+ * ----------
+ */
+void
+AfterTriggerCancelQuery(void)
+{
+ /* Set to a value denoting that no query is active. */
+ afterTriggers.query_depth = -1;
+}
+
/* ----------
* AfterTriggerEndQuery()
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 66f1b7398d..383ebee008 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,14 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
+
+ /* Mark execution as canceled if plan won't be executed. */
+ estate->es_canceled = !plan_valid;
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -325,6 +338,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
estate = queryDesc->estate;
Assert(estate != NULL);
+ Assert(!estate->es_canceled);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -429,7 +443,7 @@ standard_ExecutorFinish(QueryDesc *queryDesc)
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/* This should be run once and only once per Executor instance */
- Assert(!estate->es_finished);
+ Assert(!estate->es_finished && !estate->es_canceled);
/* Switch into per-query memory context */
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
@@ -488,11 +502,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
Assert(estate != NULL);
/*
- * Check that ExecutorFinish was called, unless in EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
- Assert(estate->es_finished ||
+ Assert(estate->es_finished || estate->es_canceled ||
(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -506,6 +520,14 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
UnregisterSnapshot(estate->es_snapshot);
UnregisterSnapshot(estate->es_crosscheck_snapshot);
+ /*
+ * Cancel trigger execution too if the query execution was canceled.
+ */
+ if (estate->es_canceled &&
+ !(estate->es_top_eflags &
+ (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
+ AfterTriggerCancelQuery();
+
/*
* Must switch out of context before destroying it
*/
@@ -829,9 +851,12 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -1014,9 +1039,15 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
}
+ queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
+ queryDesc->planstate = planstate;
+ return true;
+
plan_init_suspended:
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
+ return false;
}
/*
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..f84a3a17d5 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1430,7 +1430,8 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c3f7279b06..da8a1511ac 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -151,6 +151,7 @@ CreateExecutorState(void)
estate->es_top_eflags = 0;
estate->es_instrument = 0;
estate->es_finished = false;
+ estate->es_canceled = false;
estate->es_exprcontexts = NIL;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..8cf0b3132d 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -862,7 +862,9 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..6a96d7fc22 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2674,8 +2686,23 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2877,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2923,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e4756f8be2..204002cff2 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..9a96b77f1e 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -116,86 +111,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -426,19 +341,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -489,8 +406,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
portal->sourceText,
@@ -501,30 +418,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +474,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +496,81 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,19 +583,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1193,7 +1183,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1204,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1224,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1252,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1339,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..37554727ee 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 430e3ca7dd..d4f7c29301 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -257,6 +257,7 @@ extern void ExecASTruncateTriggers(EState *estate,
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
+extern void AfterTriggerCancelQuery(void);
extern void AfterTriggerEndQuery(EState *estate);
extern void AfterTriggerFireDeferred(void);
extern void AfterTriggerEndXact(bool isCommit);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 72cbf120c5..10c5cda169 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b2a576b76d..0922be6678 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -670,6 +670,9 @@ typedef struct EState
int es_top_eflags; /* eflags passed to ExecutorStart */
int es_instrument; /* OR of InstrumentOption flags */
bool es_finished; /* true when ExecutorFinish is done */
+ bool es_canceled; /* true when execution was canceled
+ * upon encountering that plan was invalided
+ * during ExecInitNode() */
List *es_exprcontexts; /* List of ExprContexts within EState */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v47-0003-Support-for-ExecInitNode-to-detect-CachedPlan-in.patch (36.7K, 7-v47-0003-Support-for-ExecInitNode-to-detect-CachedPlan-in.patch)
download | inline diff:
From 16f2dea62bcff534a41f3185c349248498d01383 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:34 +0900
Subject: [PATCH v47 3/8] Support for ExecInitNode() to detect CachedPlan
invalidation
This commit adds checks to determine if a CachedPlan remains valid
during ExecInitNode() traversal of the plan from the CachedPlan. This
includes points right after opening/locking tables and during
recursive ExecInitNode() calls to initialize child plans. Depending
on the situation, specific ExecInit*() routines will:
* Return NULL if invalidation is spotted right after opening a table
or after a function that opens one, but before initializing child
nodes.
* Return the partially initialized PlanState node if invalidation is
found after recursively initializing a child node via
ExecInitNode().
A prior commit already fortified ExecEnd*() to manage these partially
initialized nodes.
Importantly, this commit doesn't alter functionality. The CachedPlan
isn't fed to the executor as of now, and the executor doesn't lock
tables.
Reviewed-by: Robert Haas
---
contrib/postgres_fdw/postgres_fdw.c | 4 ++++
src/backend/executor/execMain.c | 24 ++++++++++++++++++++--
src/backend/executor/execPartition.c | 4 ++++
src/backend/executor/execProcnode.c | 19 ++++++++++++++++-
src/backend/executor/execUtils.c | 2 ++
src/backend/executor/nodeAgg.c | 2 ++
src/backend/executor/nodeAppend.c | 14 ++++++++++---
src/backend/executor/nodeBitmapAnd.c | 11 +++++++---
src/backend/executor/nodeBitmapHeapscan.c | 4 ++++
src/backend/executor/nodeBitmapIndexscan.c | 2 ++
src/backend/executor/nodeBitmapOr.c | 11 +++++++---
src/backend/executor/nodeCustom.c | 2 ++
src/backend/executor/nodeForeignscan.c | 4 ++++
src/backend/executor/nodeGather.c | 3 +++
src/backend/executor/nodeGatherMerge.c | 2 ++
src/backend/executor/nodeGroup.c | 2 ++
src/backend/executor/nodeHash.c | 2 ++
src/backend/executor/nodeHashjoin.c | 4 ++++
src/backend/executor/nodeIncrementalSort.c | 2 ++
src/backend/executor/nodeIndexonlyscan.c | 4 ++++
src/backend/executor/nodeIndexscan.c | 4 ++++
src/backend/executor/nodeLimit.c | 2 ++
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 2 ++
src/backend/executor/nodeMemoize.c | 2 ++
src/backend/executor/nodeMergeAppend.c | 10 ++++++++-
src/backend/executor/nodeMergejoin.c | 4 ++++
src/backend/executor/nodeModifyTable.c | 7 +++++++
src/backend/executor/nodeNestloop.c | 4 ++++
src/backend/executor/nodeProjectSet.c | 2 ++
src/backend/executor/nodeRecursiveunion.c | 4 ++++
src/backend/executor/nodeResult.c | 2 ++
src/backend/executor/nodeSamplescan.c | 2 ++
src/backend/executor/nodeSeqscan.c | 2 ++
src/backend/executor/nodeSetOp.c | 2 ++
src/backend/executor/nodeSort.c | 2 ++
src/backend/executor/nodeSubqueryscan.c | 2 ++
src/backend/executor/nodeTidrangescan.c | 2 ++
src/backend/executor/nodeTidscan.c | 2 ++
src/backend/executor/nodeUnique.c | 2 ++
src/backend/executor/nodeWindowAgg.c | 2 ++
src/include/executor/executor.h | 10 +++++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 +++++++++++++
44 files changed, 198 insertions(+), 13 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1393716587..ab7ecb925c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2660,7 +2660,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..66f1b7398d 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -839,8 +839,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
- TupleDesc tupType;
+ PlanState *planstate = NULL;
+ TupleDesc tupType = NULL;
ListCell *l;
int i;
@@ -855,6 +855,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = NULL;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +887,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -956,6 +959,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
i++;
}
@@ -966,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1007,6 +1014,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
}
+plan_init_suspended:
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
}
@@ -2945,6 +2953,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we shouldn't have received any new invalidation
+ * messages that would make the plan tree stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2988,6 +3002,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we shouldn't have received any new invalidation messages
+ * that would make the plan tree stale.
+ */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..e88455368c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1801,6 +1801,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1929,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 6098cdca69..169a52b038 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,7 +135,20 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
+ *
+ * The node type-specific ExecInit* routines listed in this function can
+ * either return NULL or a partially initialized PlanState tree when they
+ * detect that the CachedPlan has been invalidated. This is determined by
+ * invoking ExecPlanStillValid() at key intervals, for instance, right
+ * after opening/locking a relation, or following the call to a function
+ * that might open/lock a relation. The latter involves recursive calls
+ * to ExecInitNode() for child node initialization. If an ExecInit*
+ * routine gets a false from ExecPlanStillValid(), it should:
+ * - Return NULL if no child node was initialized at the time of
+ * checking.
+ * - Provide the partially initialized PlanState node if any child node
+ * was set up recursively by then.
* ------------------------------------------------------------------------
*/
PlanState *
@@ -388,6 +401,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ return result;
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 16704c0c2f..c3f7279b06 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -822,6 +822,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index f154f28902..b70e8c2cd6 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return aggstate;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..588f5388c7 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -147,6 +147,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -185,8 +187,13 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ /*
+ * Any uninitialized sunbodes will have NULL in appendplans in the case of
+ * an early return.
+ */
+ appendstate->appendplans = appendplanstates =
+ (PlanState **) palloc0(nplans * sizeof(PlanState *));
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +228,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..c0495ec90f 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -69,6 +69,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
*/
nplans = list_length(node->bitmapplans);
+ /*
+ * Any uninitialized sunbodes will have NULL in bitmapplans in the case of
+ * an early return.
+ */
bitmapplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
/*
@@ -78,7 +82,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
bitmapandstate->ps.state = estate;
bitmapandstate->ps.ExecProcNode = ExecBitmapAnd;
bitmapandstate->bitmapplans = bitmapplanstates;
- bitmapandstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -88,8 +91,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmapandstate;
+ bitmapandstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c8c1c9d88e..715c111be9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -752,11 +752,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 7cf8532bc9..4200472d02 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -255,6 +255,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return indexstate;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..00120669a5 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -70,6 +70,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
*/
nplans = list_length(node->bitmapplans);
+ /*
+ * Any uninitialized sunbodes will have NULL in bitmapplans in the case of
+ * an early return.
+ */
bitmapplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
/*
@@ -79,7 +83,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
bitmaporstate->ps.state = estate;
bitmaporstate->ps.ExecProcNode = ExecBitmapOr;
bitmaporstate->bitmapplans = bitmapplanstates;
- bitmaporstate->nplans = nplans;
/*
* call ExecInitNode on each of the plans to be executed and save the
@@ -89,8 +92,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return bitmaporstate;
+ bitmaporstate->nplans = i;
}
/*
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index 28b5bb9353..160eeee071 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 298ea59a1e..0f29b03977 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index bb2500a469..6b26e03f74 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gatherstate;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 7a71a58509..84412f94bb 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 8c650f0e46..b6068887f6 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return grpstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index e72f0986c2..030bf0ed43 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index aea44a9d56..49a6ba4276 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 544e64dfab..e83feae353 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return incrsortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f1db35665c..ea7fd89c0c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -496,6 +496,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -549,6 +551,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 14b9c00217..906358011a 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -909,6 +909,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -954,6 +956,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 5654158e3e..6760de0f25 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return limitstate;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..2599332f01 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 753ea28915..b974ebdc8a 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 81f2acde5e..ac0a8a0ae4 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..c9d406c230 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -95,6 +95,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -120,7 +122,11 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ /*
+ * Any uninitialized sunbodes will have NULL in mergeplans in the case of
+ * an early return.
+ */
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +157,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 648fdd9a5f..e7f4512419 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d21a178ad5..c28d5058e9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3985,6 +3985,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4012,6 +4015,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4039,6 +4044,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index fc8f833d8b..0158a3e592 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return nlstate;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index b4bbdc89b1..1b4774d4f7 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return state;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index 54cd6f2347..6398475c62 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index e9f5732f33..d4ea101cbe 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return resstate;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 41c1ea37ad..5bec5c1f64 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 49a5933aff..48e20aa735 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 98c1b84d43..7a3a142204 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index eea7f2ae15..3ebbc46604 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return sortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 1ee6295660..3c5c7c2ebb 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index da622d3f5f..d337f3d54a 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -374,6 +374,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 15055077d0..9637f354b2 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -517,6 +517,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 01f951197c..28630e380e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index f5170799e4..29151fe44b 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2461,6 +2461,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index aeebe0e0ff..72cbf120c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,15 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the CachedPlan in es_cachedplan still valid?
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..b2a576b76d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..0a9e041d51 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Invoked by the executor for each relation lock acquired during the
+ * initialization of the plan tree within the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
[application/octet-stream] v47-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch (5.9K, 8-v47-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch)
download | inline diff:
From 13e4d0b8d381c1d0e507a09e2350150d4f1bc671 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:16 +0900
Subject: [PATCH v47 2/8] Check pointer NULLness before cleanup in ExecEnd*
routines
Many routines already perform this check, but a few instances remain.
Currently, these NULLness checks might seem redundant since ExecEnd*
routines operate under the assumption that their matching ExecInit*
routine would have fully executed, ensuring pointers are set. However,
a forthcoming patch will modify ExecInit* routines to sometimes exit
early, potentially leaving some pointers in an undetermined state.
Reviewed-by: Robert Haas
---
src/backend/executor/nodeBitmapHeapscan.c | 3 ++-
src/backend/executor/nodeForeignscan.c | 13 +++++++-----
src/backend/executor/nodeIncrementalSort.c | 6 ++++--
src/backend/executor/nodeMemoize.c | 1 +
src/backend/executor/nodeRecursiveunion.c | 6 ++++--
src/backend/executor/nodeWindowAgg.c | 24 ++++++++++++++--------
6 files changed, 35 insertions(+), 18 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2db0acfc76..c8c1c9d88e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -681,7 +681,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc != NULL)
+ table_endscan(scanDesc);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 73913ebb18..298ea59a1e 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -301,13 +301,16 @@ ExecEndForeignScan(ForeignScanState *node)
EState *estate = node->ss.ps.state;
/* Let the FDW shut down */
- if (plan->operation != CMD_SELECT)
+ if (node->fdwroutine != NULL)
{
- if (estate->es_epq_active == NULL)
- node->fdwroutine->EndDirectModify(node);
+ if (plan->operation != CMD_SELECT)
+ {
+ if (estate->es_epq_active == NULL)
+ node->fdwroutine->EndDirectModify(node);
+ }
+ else
+ node->fdwroutine->EndForeignScan(node);
}
- else
- node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index cd094a190c..544e64dfab 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,8 +1079,10 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ if (node->group_pivot != NULL)
+ ExecDropSingleTupleTableSlot(node->group_pivot);
+ if (node->transfer_tuple != NULL)
+ ExecDropSingleTupleTableSlot(node->transfer_tuple);
/*
* Release tuplesort resources.
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 94bf479287..81f2acde5e 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1043,6 +1043,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable != NULL)
{
int count;
uint64 mem = 0;
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..54cd6f2347 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -272,8 +272,10 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table != NULL)
+ tuplestore_end(node->working_table);
+ if (node->intermediate_table != NULL)
+ tuplestore_end(node->intermediate_table);
/* free subsidiary stuff including hashtable */
if (node->tempContext)
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 77724a6daa..f5170799e4 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -1351,11 +1351,14 @@ release_partition(WindowAggState *winstate)
* any aggregate temp data). We don't rely on retail pfree because some
* aggregates might have allocated data we don't have direct pointers to.
*/
- MemoryContextResetAndDeleteChildren(winstate->partcontext);
- MemoryContextResetAndDeleteChildren(winstate->aggcontext);
+ if (winstate->partcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->partcontext);
+ if (winstate->aggcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->aggcontext);
for (i = 0; i < winstate->numaggs; i++)
{
- if (winstate->peragg[i].aggcontext != winstate->aggcontext)
+ if (winstate->peragg[i].aggcontext != NULL &&
+ winstate->peragg[i].aggcontext != winstate->aggcontext)
MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext);
}
@@ -2688,14 +2691,19 @@ ExecEndWindowAgg(WindowAggState *node)
for (i = 0; i < node->numaggs; i++)
{
- if (node->peragg[i].aggcontext != node->aggcontext)
+ if (node->peragg[i].aggcontext != NULL &&
+ node->peragg[i].aggcontext != node->aggcontext)
MemoryContextDelete(node->peragg[i].aggcontext);
}
- MemoryContextDelete(node->partcontext);
- MemoryContextDelete(node->aggcontext);
+ if (node->partcontext != NULL)
+ MemoryContextDelete(node->partcontext);
+ if (node->aggcontext != NULL)
+ MemoryContextDelete(node->aggcontext);
- pfree(node->perfunc);
- pfree(node->peragg);
+ if (node->perfunc != NULL)
+ pfree(node->perfunc);
+ if (node->peragg != NULL)
+ pfree(node->peragg);
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
--
2.35.3
[application/octet-stream] v47-0001-Remove-obsolete-executor-cleanup-code.patch (30.9K, 9-v47-0001-Remove-obsolete-executor-cleanup-code.patch)
download | inline diff:
From ecc53680d6e685c8ad1efdb18c8d432751c142dc Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:52:39 +0900
Subject: [PATCH v47 1/8] Remove obsolete executor cleanup code
This commit removes unnecessary ExecExprFreeContext() calls in ExecEnd*
routines as the actual cleanup is managed by FreeExecutorState. With
no remaining callers for ExecExprFreeContext(), this commit also
removes the function.
This commit also drops redundant ExecClearTuple() calls, as
ExecResetTupleTable() in ExecEndPlan() already takes care of resetting
all TupleTableSlots.
After these modifications, the ExecEnd*() routines for ValuesScan,
NamedTuplestoreScan, and WorkTableScan became redundant. Thus, this
commit removes them. These changes not only optimize CPU usage during
ExecEndNode() processing but also pave the way for an upcoming patch.
This future patch aims to allow ExecEndNode() to expect PlanState
trees that are only partially initialized in some cases.
Reviewed-by: Robert Haas
---
src/backend/executor/execProcnode.c | 18 +++++--------
src/backend/executor/execUtils.c | 26 -------------------
src/backend/executor/nodeAgg.c | 10 -------
src/backend/executor/nodeBitmapHeapscan.c | 12 ---------
src/backend/executor/nodeBitmapIndexscan.c | 8 ------
src/backend/executor/nodeCtescan.c | 12 ---------
src/backend/executor/nodeCustom.c | 7 -----
src/backend/executor/nodeForeignscan.c | 8 ------
src/backend/executor/nodeFunctionscan.c | 12 ---------
src/backend/executor/nodeGather.c | 3 ---
src/backend/executor/nodeGatherMerge.c | 3 ---
src/backend/executor/nodeGroup.c | 5 ----
src/backend/executor/nodeHash.c | 5 ----
src/backend/executor/nodeHashjoin.c | 12 ---------
src/backend/executor/nodeIncrementalSort.c | 5 ----
src/backend/executor/nodeIndexonlyscan.c | 16 ------------
src/backend/executor/nodeIndexscan.c | 16 ------------
src/backend/executor/nodeLimit.c | 1 -
src/backend/executor/nodeMaterial.c | 5 ----
src/backend/executor/nodeMemoize.c | 9 -------
src/backend/executor/nodeMergejoin.c | 12 ---------
src/backend/executor/nodeModifyTable.c | 11 --------
.../executor/nodeNamedtuplestorescan.c | 22 ----------------
src/backend/executor/nodeNestloop.c | 11 --------
src/backend/executor/nodeProjectSet.c | 10 -------
src/backend/executor/nodeResult.c | 10 -------
src/backend/executor/nodeSamplescan.c | 12 ---------
src/backend/executor/nodeSeqscan.c | 12 ---------
src/backend/executor/nodeSetOp.c | 4 ---
src/backend/executor/nodeSort.c | 7 -----
src/backend/executor/nodeSubqueryscan.c | 12 ---------
src/backend/executor/nodeTableFuncscan.c | 12 ---------
src/backend/executor/nodeTidrangescan.c | 12 ---------
src/backend/executor/nodeTidscan.c | 12 ---------
src/backend/executor/nodeUnique.c | 5 ----
src/backend/executor/nodeValuesscan.c | 24 -----------------
src/backend/executor/nodeWindowAgg.c | 17 ------------
src/backend/executor/nodeWorktablescan.c | 22 ----------------
src/include/executor/executor.h | 1 -
.../executor/nodeNamedtuplestorescan.h | 1 -
src/include/executor/nodeValuesscan.h | 1 -
src/include/executor/nodeWorktablescan.h | 1 -
42 files changed, 6 insertions(+), 418 deletions(-)
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..6098cdca69 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -667,22 +667,10 @@ ExecEndNode(PlanState *node)
ExecEndTableFuncScan((TableFuncScanState *) node);
break;
- case T_ValuesScanState:
- ExecEndValuesScan((ValuesScanState *) node);
- break;
-
case T_CteScanState:
ExecEndCteScan((CteScanState *) node);
break;
- case T_NamedTuplestoreScanState:
- ExecEndNamedTuplestoreScan((NamedTuplestoreScanState *) node);
- break;
-
- case T_WorkTableScanState:
- ExecEndWorkTableScan((WorkTableScanState *) node);
- break;
-
case T_ForeignScanState:
ExecEndForeignScan((ForeignScanState *) node);
break;
@@ -757,6 +745,12 @@ ExecEndNode(PlanState *node)
ExecEndLimit((LimitState *) node);
break;
+ /* No clean up actions for these nodes. */
+ case T_ValuesScanState:
+ case T_NamedTuplestoreScanState:
+ case T_WorkTableScanState:
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(node));
break;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..16704c0c2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -638,32 +638,6 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, int varno, TupleDesc tupdesc)
return true;
}
-/* ----------------
- * ExecFreeExprContext
- *
- * A plan node's ExprContext should be freed explicitly during executor
- * shutdown because there may be shutdown callbacks to call. (Other resources
- * made by the above routines, such as projection info, don't need to be freed
- * explicitly because they're just memory in the per-query memory context.)
- *
- * However ... there is no particular need to do it during ExecEndNode,
- * because FreeExecutorState will free any remaining ExprContexts within
- * the EState. Letting FreeExecutorState do it allows the ExprContexts to
- * be freed in reverse order of creation, rather than order of creation as
- * will happen if we delete them here, which saves O(N^2) work in the list
- * cleanup inside FreeExprContext.
- * ----------------
- */
-void
-ExecFreeExprContext(PlanState *planstate)
-{
- /*
- * Per above discussion, don't actually delete the ExprContext. We do
- * unlink it from the plan node, though.
- */
- planstate->ps_ExprContext = NULL;
-}
-
/* ----------------------------------------------------------------
* Scan node support
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..f154f28902 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4357,16 +4357,6 @@ ExecEndAgg(AggState *node)
if (node->hashcontext)
ReScanExprContext(node->hashcontext);
- /*
- * We don't actually free any ExprContexts here (see comment in
- * ExecFreeExprContext), just unlinking the output one from the plan node
- * suffices.
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..2db0acfc76 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -655,18 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..7cf8532bc9 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -184,14 +184,6 @@ ExecEndBitmapIndexScan(BitmapIndexScanState *node)
indexRelationDesc = node->biss_RelationDesc;
indexScanDesc = node->biss_ScanDesc;
- /*
- * Free the exprcontext ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- if (node->biss_RuntimeContext)
- FreeExprContext(node->biss_RuntimeContext, true);
-#endif
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index cc4c4243e2..a0c0c4be33 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -287,18 +287,6 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
void
ExecEndCteScan(CteScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* If I am the leader, free the tuplestore.
*/
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..28b5bb9353 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -129,13 +129,6 @@ ExecEndCustomScan(CustomScanState *node)
{
Assert(node->methods->EndCustomScan != NULL);
node->methods->EndCustomScan(node);
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* Clean out the tuple table */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..73913ebb18 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -312,14 +312,6 @@ ExecEndForeignScan(ForeignScanState *node)
/* Shut down any outer plan. */
if (outerPlanState(node))
ExecEndNode(outerPlanState(node));
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean out the tuple table */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index dd06ef8aee..a49c1a2c85 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -523,18 +523,6 @@ ExecEndFunctionScan(FunctionScanState *node)
{
int i;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release slots and tuplestore resources
*/
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..bb2500a469 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -250,9 +250,6 @@ ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..7a71a58509 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -290,9 +290,6 @@ ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..8c650f0e46 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -228,11 +228,6 @@ ExecEndGroup(GroupState *node)
{
PlanState *outerPlan;
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..e72f0986c2 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -415,11 +415,6 @@ ExecEndHash(HashState *node)
{
PlanState *outerPlan;
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..aea44a9d56 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -867,18 +867,6 @@ ExecEndHashJoin(HashJoinState *node)
node->hj_HashTable = NULL;
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->hj_OuterTupleSlot);
- ExecClearTuple(node->hj_HashTupleSlot);
-
/*
* clean up subtrees
*/
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..cd094a190c 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,11 +1079,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- /* clean out the scan tuple */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- /* must drop standalone tuple slots from outer node */
ExecDropSingleTupleTableSlot(node->group_pivot);
ExecDropSingleTupleTableSlot(node->transfer_tuple);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..f1db35665c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -380,22 +380,6 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->ioss_RuntimeContext)
- FreeExprContext(node->ioss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..14b9c00217 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -794,22 +794,6 @@ ExecEndIndexScan(IndexScanState *node)
indexRelationDesc = node->iss_RelationDesc;
indexScanDesc = node->iss_ScanDesc;
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->iss_RuntimeContext)
- FreeExprContext(node->iss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..5654158e3e 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -534,7 +534,6 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
void
ExecEndLimit(LimitState *node)
{
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..753ea28915 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -239,11 +239,6 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
void
ExecEndMaterial(MaterialState *node)
{
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..94bf479287 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1091,15 +1091,6 @@ ExecEndMemoize(MemoizeState *node)
/* Remove the cache context */
MemoryContextDelete(node->tableContext);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to cache result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..648fdd9a5f 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1642,18 +1642,6 @@ ExecEndMergeJoin(MergeJoinState *node)
{
MJ1_printf("ExecEndMergeJoin: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->mj_MarkedTupleSlot);
-
/*
* shut down the subplans
*/
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5005d8c0d1..d21a178ad5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4446,17 +4446,6 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* Terminate EPQ execution if active
*/
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index 46832ad82f..3547dc2b10 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -155,28 +155,6 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndNamedTuplestoreScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanNamedTuplestoreScan
*
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..fc8f833d8b 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -363,17 +363,6 @@ ExecEndNestLoop(NestLoopState *node)
{
NL1_printf("ExecEndNestLoop: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..b4bbdc89b1 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -320,16 +320,6 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
void
ExecEndProjectSet(ProjectSetState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..e9f5732f33 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -240,16 +240,6 @@ ExecInitResult(Result *node, EState *estate, int eflags)
void
ExecEndResult(ResultState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..41c1ea37ad 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -188,18 +188,6 @@ ExecEndSampleScan(SampleScanState *node)
if (node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..49a5933aff 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -190,18 +190,6 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..98c1b84d43 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -582,13 +582,9 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
void
ExecEndSetOp(SetOpState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/* free subsidiary stuff including hashtable */
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..eea7f2ae15 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -303,13 +303,6 @@ ExecEndSort(SortState *node)
SO1_printf("ExecEndSort: %s\n",
"shutting down sort node");
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
/*
* Release tuplesort resources
*/
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..1ee6295660 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -167,18 +167,6 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
void
ExecEndSubqueryScan(SubqueryScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the upper tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subquery
*/
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 791cbd2372..a60dcd4943 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -213,18 +213,6 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
void
ExecEndTableFuncScan(TableFuncScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..da622d3f5f 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -331,18 +331,6 @@ ExecEndTidRangeScan(TidRangeScanState *node)
if (scan != NULL)
table_endscan(scan);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..15055077d0 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -472,18 +472,6 @@ ExecEndTidScan(TidScanState *node)
{
if (node->ss.ss_currentScanDesc)
table_endscan(node->ss.ss_currentScanDesc);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..01f951197c 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -168,11 +168,6 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
void
ExecEndUnique(UniqueState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- ExecFreeExprContext(&node->ps);
-
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index 32ace63017..fbfb067f3b 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -319,30 +319,6 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndValuesScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndValuesScan(ValuesScanState *node)
-{
- /*
- * Free both exprcontexts
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->rowcontext;
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanValuesScan
*
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..77724a6daa 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2686,23 +2686,6 @@ ExecEndWindowAgg(WindowAggState *node)
release_partition(node);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- ExecClearTuple(node->first_part_slot);
- ExecClearTuple(node->agg_row_slot);
- ExecClearTuple(node->temp_slot_1);
- ExecClearTuple(node->temp_slot_2);
- if (node->framehead_slot)
- ExecClearTuple(node->framehead_slot);
- if (node->frametail_slot)
- ExecClearTuple(node->frametail_slot);
-
- /*
- * Free both the expr contexts.
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->tmpcontext;
- ExecFreeExprContext(&node->ss.ps);
-
for (i = 0; i < node->numaggs; i++)
{
if (node->peragg[i].aggcontext != node->aggcontext)
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 0c13448236..17a548865e 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -181,28 +181,6 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndWorkTableScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndWorkTableScan(WorkTableScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanWorkTableScan
*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..aeebe0e0ff 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -569,7 +569,6 @@ extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
-extern void ExecFreeExprContext(PlanState *planstate);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/nodeNamedtuplestorescan.h b/src/include/executor/nodeNamedtuplestorescan.h
index 3ff687023a..9d80236fe5 100644
--- a/src/include/executor/nodeNamedtuplestorescan.h
+++ b/src/include/executor/nodeNamedtuplestorescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern NamedTuplestoreScanState *ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflags);
-extern void ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node);
extern void ExecReScanNamedTuplestoreScan(NamedTuplestoreScanState *node);
#endif /* NODENAMEDTUPLESTORESCAN_H */
diff --git a/src/include/executor/nodeValuesscan.h b/src/include/executor/nodeValuesscan.h
index a52fa678df..fe3f043951 100644
--- a/src/include/executor/nodeValuesscan.h
+++ b/src/include/executor/nodeValuesscan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern ValuesScanState *ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags);
-extern void ExecEndValuesScan(ValuesScanState *node);
extern void ExecReScanValuesScan(ValuesScanState *node);
#endif /* NODEVALUESSCAN_H */
diff --git a/src/include/executor/nodeWorktablescan.h b/src/include/executor/nodeWorktablescan.h
index e553a453f3..f31b22cec4 100644
--- a/src/include/executor/nodeWorktablescan.h
+++ b/src/include/executor/nodeWorktablescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern WorkTableScanState *ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags);
-extern void ExecEndWorkTableScan(WorkTableScanState *node);
extern void ExecReScanWorkTableScan(WorkTableScanState *node);
#endif /* NODEWORKTABLESCAN_H */
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-06 14:20 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Robert Haas @ 2023-09-06 14:20 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Wed, Sep 6, 2023 at 5:12 AM Amit Langote <[email protected]> wrote:
> Attached updated patches. Thanks for the review.
I think 0001 looks ready to commit. I'm not sure that the commit
message needs to mention future patches here, since this code cleanup
seems like a good idea regardless, but if you feel otherwise, fair
enough.
On 0002, some questions:
- In ExecEndLockRows, is the call to EvalPlanQualEnd a concern? i.e.
Does that function need any adjustment?
- In ExecEndMemoize, should there be a null-test around
MemoryContextDelete(node->tableContext) as we have in
ExecEndRecursiveUnion, ExecEndSetOp, etc.?
I wonder how we feel about setting pointers to NULL after freeing the
associated data structures. The existing code isn't consistent about
doing that, and making it do so would be a fairly large change that
would bloat this patch quite a bit. On the other hand, I think it's a
good practice as a general matter, and we do do it in some ExecEnd
functions.
On 0003, I have some doubt about whether we really have all the right
design decisions in detail here:
- Why have this weird rule where sometimes we return NULL and other
times the planstate? Is there any point to such a coding rule? Why not
just always return the planstate?
- Is there any point to all of these early exit cases? For example, in
ExecInitBitmapAnd, why exit early if initialization fails? Why not
just plunge ahead and if initialization failed the caller will notice
that and when we ExecEndNode some of the child node pointers will be
NULL but who cares? The obvious disadvantage of this approach is that
we're doing a bunch of unnecessary initialization, but we're also
speeding up the common case where we don't need to abort by avoiding a
branch that will rarely be taken. I'm not quite sure what the right
thing to do is here.
- The cases where we call ExecGetRangeTableRelation or
ExecOpenScanRelation are a bit subtler ... maybe initialization that
we're going to do later is going to barf if the tuple descriptor of
the relation isn't what we thought it was going to be. In that case it
becomes important to exit early. But if that's not actually a problem,
then we could apply the same principle here also -- don't pollute the
code with early-exit cases, just let it do its thing and sort it out
later. Do you know what the actual problems would be here if we didn't
exit early in these cases?
- Depending on the answers to the above points, one thing we could
think of doing is put an early exit case into ExecInitNode itself: if
(unlikely(!ExecPlanStillValid(whatever)) return NULL. Maybe Andres or
someone is going to argue that that checks too often and is thus too
expensive, but it would be a lot more maintainable than having similar
checks strewn throughout the ExecInit* functions. Perhaps it deserves
some thought/benchmarking. More generally, if there's anything we can
do to centralize these checks in fewer places, I think that would be
worth considering. The patch isn't terribly large as it stands, so I
don't necessarily think that this is a critical issue, but I'm just
wondering if we can do better. I'm not even sure that it would be too
expensive to just initialize the whole plan always, and then just do
one test at the end. That's not OK if the changed tuple descriptor (or
something else) is going to crash or error out in a funny way or
something before initialization is completed, but if it's just going
to result in burning a few CPU cycles in a corner case, I don't know
if we should really care.
- The "At this point" comments don't give any rationale for why we
shouldn't have received any such invalidation messages. That makes
them fairly useless; the Assert by itself clarifies that you think
that case shouldn't happen. The comment's job is to justify that
claim.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-25 12:57 Amit Langote <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-09-25 12:57 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Wed, Sep 6, 2023 at 11:20 PM Robert Haas <[email protected]> wrote:
> On Wed, Sep 6, 2023 at 5:12 AM Amit Langote <[email protected]> wrote:
> > Attached updated patches. Thanks for the review.
>
> I think 0001 looks ready to commit. I'm not sure that the commit
> message needs to mention future patches here, since this code cleanup
> seems like a good idea regardless, but if you feel otherwise, fair
> enough.
OK, I will remove the mention of future patches.
> On 0002, some questions:
>
> - In ExecEndLockRows, is the call to EvalPlanQualEnd a concern? i.e.
> Does that function need any adjustment?
I think it does with the patch as it stands. It needs to have an
early exit at the top if parentestate is NULL, which it would be if
EvalPlanQualInit() wasn't called from an ExecInit*() function.
Though, as I answer below your question as to whether there is
actually any need to interrupt all of the ExecInit*() routines,
nothing needs to change in ExecEndLockRows().
> - In ExecEndMemoize, should there be a null-test around
> MemoryContextDelete(node->tableContext) as we have in
> ExecEndRecursiveUnion, ExecEndSetOp, etc.?
Oops, you're right. Added.
> I wonder how we feel about setting pointers to NULL after freeing the
> associated data structures. The existing code isn't consistent about
> doing that, and making it do so would be a fairly large change that
> would bloat this patch quite a bit. On the other hand, I think it's a
> good practice as a general matter, and we do do it in some ExecEnd
> functions.
I agree that it might be worthwhile to take the opportunity and make
the code more consistent in this regard. So, I've included those
changes too in 0002.
> On 0003, I have some doubt about whether we really have all the right
> design decisions in detail here:
>
> - Why have this weird rule where sometimes we return NULL and other
> times the planstate? Is there any point to such a coding rule? Why not
> just always return the planstate?
>
> - Is there any point to all of these early exit cases? For example, in
> ExecInitBitmapAnd, why exit early if initialization fails? Why not
> just plunge ahead and if initialization failed the caller will notice
> that and when we ExecEndNode some of the child node pointers will be
> NULL but who cares? The obvious disadvantage of this approach is that
> we're doing a bunch of unnecessary initialization, but we're also
> speeding up the common case where we don't need to abort by avoiding a
> branch that will rarely be taken. I'm not quite sure what the right
> thing to do is here.
>
> - The cases where we call ExecGetRangeTableRelation or
> ExecOpenScanRelation are a bit subtler ... maybe initialization that
> we're going to do later is going to barf if the tuple descriptor of
> the relation isn't what we thought it was going to be. In that case it
> becomes important to exit early. But if that's not actually a problem,
> then we could apply the same principle here also -- don't pollute the
> code with early-exit cases, just let it do its thing and sort it out
> later. Do you know what the actual problems would be here if we didn't
> exit early in these cases?
>
> - Depending on the answers to the above points, one thing we could
> think of doing is put an early exit case into ExecInitNode itself: if
> (unlikely(!ExecPlanStillValid(whatever)) return NULL. Maybe Andres or
> someone is going to argue that that checks too often and is thus too
> expensive, but it would be a lot more maintainable than having similar
> checks strewn throughout the ExecInit* functions. Perhaps it deserves
> some thought/benchmarking. More generally, if there's anything we can
> do to centralize these checks in fewer places, I think that would be
> worth considering. The patch isn't terribly large as it stands, so I
> don't necessarily think that this is a critical issue, but I'm just
> wondering if we can do better. I'm not even sure that it would be too
> expensive to just initialize the whole plan always, and then just do
> one test at the end. That's not OK if the changed tuple descriptor (or
> something else) is going to crash or error out in a funny way or
> something before initialization is completed, but if it's just going
> to result in burning a few CPU cycles in a corner case, I don't know
> if we should really care.
I thought about this some and figured that adding the
is-CachedPlan-still-valid tests in the following places should suffice
after all:
1. In InitPlan() right after the top-level ExecInitNode() calls
2. In ExecInit*() functions of Scan nodes, right after
ExecOpenScanRelation() calls
CachedPlans can only become invalid because of concurrent changes to
the inheritance child tables referenced in the plan. Only the
following schema modifications of child tables are possible to be
performed concurrently:
* Addition of a column (allowed only if traditional inheritance child)
* Addition of an index
* Addition of a non-index constraint
* Dropping of a child table (allowed only if traditional inheritance child)
* Dropping of an index referenced in the plan
The first 3 are not destructive enough to cause crashes, weird errors
during ExecInit*(), though the last two can be, so the 2nd set of the
tests after ExecOpenScanRelation() mentioned above.
> - The "At this point" comments don't give any rationale for why we
> shouldn't have received any such invalidation messages. That makes
> them fairly useless; the Assert by itself clarifies that you think
> that case shouldn't happen. The comment's job is to justify that
> claim.
I've rewritten the comments.
I'll post the updated set of patches shortly.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-26 13:06 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-09-26 13:06 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Mon, Sep 25, 2023 at 9:57 PM Amit Langote <[email protected]> wrote:
> On Wed, Sep 6, 2023 at 11:20 PM Robert Haas <[email protected]> wrote:
> > - Is there any point to all of these early exit cases? For example, in
> > ExecInitBitmapAnd, why exit early if initialization fails? Why not
> > just plunge ahead and if initialization failed the caller will notice
> > that and when we ExecEndNode some of the child node pointers will be
> > NULL but who cares? The obvious disadvantage of this approach is that
> > we're doing a bunch of unnecessary initialization, but we're also
> > speeding up the common case where we don't need to abort by avoiding a
> > branch that will rarely be taken. I'm not quite sure what the right
> > thing to do is here.
> I thought about this some and figured that adding the
> is-CachedPlan-still-valid tests in the following places should suffice
> after all:
>
> 1. In InitPlan() right after the top-level ExecInitNode() calls
> 2. In ExecInit*() functions of Scan nodes, right after
> ExecOpenScanRelation() calls
After sleeping on this, I think we do need the checks after all the
ExecInitNode() calls too, because we have many instances of the code
like the following one:
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
<some code that dereferences outDesc>
If outerNode is a SeqScan and ExecInitSeqScan() returned early because
ExecOpenScanRelation() detected that plan was invalidated, then
tupDesc would be NULL in this case, causing the code to crash.
Now one might say that perhaps we should only add the
is-CachedPlan-valid test in the instances where there is an actual
risk of such misbehavior, but that could lead to confusion, now or
later. It seems better to add them after every ExecInitNode() call
while we're inventing the notion, because doing so relieves the
authors of future enhancements of the ExecInit*() routines from
worrying about any of this.
Attached 0003 should show how that turned out.
Updated 0002 as mentioned in the previous reply -- setting pointers to
NULL after freeing them more consistently across various ExecEnd*()
routines and using the `if (pointer != NULL)` style over the `if
(pointer)` more consistently.
Updated 0001's commit message to remove the mention of its relation to
any future commits. I intend to push it tomorrow.
Patches 0004 onwards contain changes too, mainly in terms of moving
the code around from one patch to another, but I'll omit the details
of the specific change for now.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v47-0005-Teach-the-executor-to-lock-child-tables-in-some-.patch (11.1K, 2-v47-0005-Teach-the-executor-to-lock-child-tables-in-some-.patch)
download | inline diff:
From 5ce427a754d034cb2b3efa922e8c9d7dad200418 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:17:15 +0900
Subject: [PATCH v47 5/9] Teach the executor to lock child tables in some cases
An upcoming commit will move the locking of child tables referenced
in a cached plan tree from GetCachedPlan() to the executor
initialization of the plan tree in ExecutorStart(). This commit
teaches ExecGetRangeTableRelation() to lock child tables if
EState.es_cachedplan points to a CachedPlan.
The executor must now deal with the cases where an unlocked child
table might have been concurrently dropped, so this modifies
ExecGetRangeTableRelation() to use try_table_open(). All of its
callers (and those of ExecOpenScanRelation() that calls it) must
now account for the child table disappearing, which means to abort
initializing the table's Scan node in the middle.
ExecGetRangeTableRelation() now examines inFromCl field of an RTE
to determine that a given range table relation is a child table, so
this commit also makes the planner set inFromCl to false in the
child tables' RTEs that it manufactures.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/executor/README | 36 +++++++++++++++++++++++-
src/backend/executor/execPartition.c | 2 ++
src/backend/executor/execUtils.c | 41 +++++++++++++++++++++-------
src/backend/optimizer/util/inherit.c | 7 +++++
src/backend/parser/analyze.c | 7 ++---
src/include/nodes/parsenodes.h | 8 ++++--
6 files changed, 84 insertions(+), 17 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..6d2240610d 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,34 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Typically, when the executor initializes a plan tree for execution, it doesn't
+lock non-index relations if the plan tree is freshly generated and not derived
+from a CachedPlan. This is because such locks have already been established
+during the query's parsing, rewriting, and planning phases. However, with a
+cached plan tree, there can be relations that remain unlocked. The function
+GetCachedPlan() locks relations existing in the query's range table pre-planning
+but doesn't account for those added during the planning phase. Consequently,
+inheritance child tables, introduced to the query's range table during planning,
+won't be locked when the cached plan reaches the executor.
+
+The decision to defer locking child tables with GetCachedPlan() arises from the
+fact that not all might be accessed during plan execution. For instance, if
+child tables are partitions, some might be omitted due to pruning at
+execution-initialization-time. Thus, the responsibility of locking these child
+tables is pushed to execution-initialization-time, taking place in ExecInitNode()
+for plan nodes encompassing these tables.
+
+This approach opens a window where a cached plan tree with child tables could
+become outdated if another backend modifies these tables before ExecInitNode()
+locks them. Given this, the executor has the added duty to confirm the plan
+tree's validity whenever it locks a child table post execution-initialization-
+pruning. This validation is done by checking the CachedPlan.is_valid attribute
+of the CachedPlan provided. If the plan tree is outdated (is_valid=false), the
+executor halts any further initialization and alerts the caller that they should
+retry execution with another freshly created plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +344,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..84978c5525 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1927,6 +1927,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (unlikely(partrel == NULL))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f0f5740c26..117773706a 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -697,6 +697,8 @@ ExecRelationIsTargetRelation(EState *estate, Index scanrelid)
*
* Open the heap relation to be scanned by a base-level scan plan node.
* This should be called during the node's ExecInit routine.
+ *
+ * NULL is returned if the relation is found to have been dropped.
* ----------------------------------------------------------------
*/
Relation
@@ -706,6 +708,8 @@ ExecOpenScanRelation(EState *estate, Index scanrelid, int eflags)
/* Open the relation. */
rel = ExecGetRangeTableRelation(estate, scanrelid);
+ if (unlikely(rel == NULL))
+ return NULL;
/*
* Complain if we're attempting a scan of an unscannable relation, except
@@ -763,6 +767,9 @@ ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos)
* Open the Relation for a range table entry, if not already done
*
* The Relations will be closed again in ExecEndPlan().
+ *
+ * Returned value may be NULL if the relation is a child relation that is not
+ * already locked.
*/
Relation
ExecGetRangeTableRelation(EState *estate, Index rti)
@@ -779,7 +786,28 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ *
+ * Note that we use try_table_open() here, because without a lock
+ * held on the relation, it may have disappeared from under us.
+ */
+ rel = try_table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -792,15 +820,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -823,6 +842,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (unlikely(resultRelationDesc == NULL))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..1b30c0ff87 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,13 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+
+ /*
+ * Flag child tables as indirectly referenced in the query. This helps
+ * the executor's ExecGetRangeTableRelation() recognize them as
+ * inheritance children.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 7a1dfb6364..cf269f8c53 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3305,10 +3305,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fef4c714b8..d8b5d0c502 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,15 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. The executor uses
+ * this to ascertain if an RTE_RELATION entry is for a table explicitly
+ * named in the query or a child table added by the planner. This
+ * distinction is vital when child tables in a plan must be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v47-0007-Add-field-to-store-parent-relids-to-Append-Merge.patch (26.1K, 3-v47-0007-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 3b2efe73e30649f7d2059b8a9d901b19a5986ad1 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:02 +0900
Subject: [PATCH v47 7/9] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo. A future commit will create a need for them to
be available at all times for the purpose of locking those
parent/ancestor tables when executing a cached plan, so add a
field called allpartrelids to Append/MergeAppend to store those
RT indexes. This also adds a function called
ExecLockAppendNonLeafTables() to lock those tables.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 2 +-
src/backend/executor/execUtils.c | 33 ++++++
src/backend/executor/nodeAppend.c | 14 +++
src/backend/executor/nodeMergeAppend.c | 14 +++
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/executor/executor.h | 1 +
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
13 files changed, 266 insertions(+), 124 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ffc62e379a..2804ec70f1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1475,7 +1475,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafPartitions().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 117773706a..2b7a08c9ba 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -827,6 +827,39 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafPartitions
+ * Lock non-leaf partitions whose child partitions are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i = -1;
+
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ /*
+ * Don't lock the root parent mentioned in the query, because it
+ * should already have been locked before entering the executor.
+ */
+ if (!rte->inFromCl)
+ LockRelationOid(rte->relid, rte->rellockmode);
+ else
+ Assert(CheckRelLockedByMe(rte->relid, rte->rellockmode, true));
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 53ca9dc85d..4759511f87 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,20 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 52c3edf278..158210aac1 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,20 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..d1f4f606bf 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1229,6 +1230,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1370,15 +1372,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1399,7 +1409,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1445,6 +1456,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1534,15 +1546,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1554,7 +1574,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 5700bfb5cd..c235d3488f 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 10c5cda169..74a471e3e3 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -601,6 +601,7 @@ exec_rt_fetch(Index rti, EState *estate)
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
+extern void ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids);
extern int executor_errposition(EState *estate, int location);
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v47-0009-Track-opened-range-table-relations-in-a-List-in-.patch (2.5K, 4-v47-0009-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 1bc15cf1211111a6986e097f5a7ac3899c9a784b Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:19 +0900
Subject: [PATCH v47 9/9] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 3 +++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 2804ec70f1..d559c1de61 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1649,12 +1649,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 2b7a08c9ba..1dfef44495 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -822,6 +822,9 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ if (rel != NULL)
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index bb5734edb5..8bbe1f6b14 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v47-0006-Assert-that-relations-needing-their-permissions-.patch (5.1K, 5-v47-0006-Assert-that-relations-needing-their-permissions-.patch)
download | inline diff:
From 2b3187b0443f8569ddbb3fd55cb72f1c0af34a43 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Mon, 25 Sep 2023 11:52:02 +0900
Subject: [PATCH v47 6/9] Assert that relations needing their permissions
checked are locked
---
src/backend/executor/execMain.c | 11 +++++++
src/backend/storage/lmgr/lmgr.c | 45 +++++++++++++++++++++++++++++
src/backend/utils/cache/lsyscache.c | 21 ++++++++++++++
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
5 files changed, 79 insertions(+)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5755336abd..ffc62e379a 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -626,6 +626,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
(rte->rtekind == RTE_SUBQUERY &&
rte->relkind == RELKIND_VIEW));
+ /*
+ * Relations whose permissions need to be checked must already
+ * have been locked by the parser or by GetCachedPlan() if a
+ * cached plan is being executed.
+ *
+ * XXX Maybe we should we skip calling ExecCheckPermissions from
+ * InitPlan in a parallel worker.
+ */
+ Assert(IsParallelWorker() ||
+ CheckRelLockedByMe(rte->relid, AccessShareLock, true));
+
(void) getRTEPermissionInfo(rteperminfos, rte);
/* Many-to-one mapping not allowed */
Assert(!bms_is_member(rte->perminfoindex, indexset));
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
--
2.35.3
[application/octet-stream] v47-0008-Delay-locking-of-child-tables-in-cached-plans-un.patch (25.2K, 6-v47-0008-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From a9e257693790a8b3823b598edd071de7478b6bd0 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:15 +0900
Subject: [PATCH v47 8/9] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
Previous commits have made all the necessary adjustment to make the
executor lock child tables, to detect invalidation of the CachedPlan
resulting from that, and to retry the execution with a new CachePlan.
So, this commit simply removes the code in plancache.c that does the
"for execution" locking, aka AcquireExecutorLocks().
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/spi.c | 2 +-
src/backend/tcop/pquery.c | 6 +-
src/backend/utils/cache/plancache.c | 154 +++++++----------
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 158 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
7 files changed, 343 insertions(+), 108 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 814ff1390f..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2680,7 +2680,7 @@ replan:
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
- NULL,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index fcf9925ed4..8d0772ae29 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -412,7 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
- NULL,
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -443,6 +443,7 @@ PortalStart(Portal portal, ParamListInfo params,
*/
if (!ExecutorStart(queryDesc, myeflags))
{
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
PopActiveSnapshot();
@@ -542,7 +543,7 @@ PortalStart(Portal portal, ParamListInfo params,
* PortalRunMulti() before calling ExecutorRun().
*/
queryDesc = CreateQueryDesc(plan,
- NULL,
+ portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -566,6 +567,7 @@ PortalStart(Portal portal, ParamListInfo params,
if (!ExecutorStart(queryDesc, myeflags))
{
PopActiveSnapshot();
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
plan_valid = false;
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 7d4168f82f..35d903cb98 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -104,13 +104,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -792,8 +792,13 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan includes child relations introduced by the planner, they
+ * wouldn't be locked yet. This is because AcquirePlannerLocks() only locks
+ * relations present in the original query's range table (before planner
+ * entry). Hence, the plan might become stale if child relations are modified
+ * concurrently. During the plan initialization, the executor must ensure the
+ * plan (CachedPlan) remains valid after locking each child table. If found
+ * invalid, the caller should be prompted to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -807,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1130,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * Typically, the plan returned by this function is valid. However, a caveat
+ * arises with inheritance/partition child tables. These aren't locked by
+ * this function, as we only lock tables directly mentioned in the original
+ * query here. The task of locking these child tables falls to the executor
+ * during plan tree setup. If acquiring these locks invalidates the plan, the
+ * executor should inform the caller to regenerate the plan by invoking this
+ * function again. The reason for this deferred child table locking mechanism
+ * is efficiency: not all might need to be locked. Some could be pruned during
+ * executor initialization, especially if their corresponding plan nodes
+ * facilitate partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1166,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1364,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1741,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..122d81f2ee
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,158 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+----------------------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = $1)
+(7 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(5 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..2d0607b176
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo11 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v47-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch (31.3K, 7-v47-0002-Check-pointer-NULLness-before-cleanup-in-ExecEnd.patch)
download | inline diff:
From 95af485bf11e05d49452601e0673ee3c415176dd Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:16 +0900
Subject: [PATCH v47 2/9] Check pointer NULLness before cleanup in ExecEnd*
routines
Many routines already perform this check, but a few instances remain.
Currently, these NULLness checks might seem redundant since ExecEnd*
routines operate under the assumption that their matching ExecInit*
routine would have fully executed, ensuring pointers are set. However,
a forthcoming patch will modify ExecInit* routines to sometimes exit
early, potentially leaving some pointers in an undetermined state,
so it will become crucial to have these NULLness checks in place.
Other than the ExecEnd* routines, this also adds a guard at the
begigging of EvalPlanQualEnd() to return early if the EPQState does
not appear to have been initialized.
While at it, set cleaned up pointers to NULL more consistently across
ExecEnd* routines. Also for consistency, change all the NULLness
checks to follow the "if (pointer != NULL)" style over "if (pointer)".
Reviewed-by: Robert Haas
---
contrib/postgres_fdw/postgres_fdw.c | 5 ++-
src/backend/executor/execMain.c | 4 ++
src/backend/executor/nodeAgg.c | 27 +++++++++----
src/backend/executor/nodeAppend.c | 3 ++
src/backend/executor/nodeBitmapAnd.c | 4 +-
src/backend/executor/nodeBitmapHeapscan.c | 47 +++++++++++++++-------
src/backend/executor/nodeBitmapIndexscan.c | 23 +++++------
src/backend/executor/nodeBitmapOr.c | 4 +-
src/backend/executor/nodeForeignscan.c | 17 ++++----
src/backend/executor/nodeGather.c | 1 +
src/backend/executor/nodeGatherMerge.c | 1 +
src/backend/executor/nodeGroup.c | 6 +--
src/backend/executor/nodeHash.c | 6 +--
src/backend/executor/nodeHashjoin.c | 4 +-
src/backend/executor/nodeIncrementalSort.c | 13 +++++-
src/backend/executor/nodeIndexonlyscan.c | 25 ++++++------
src/backend/executor/nodeIndexscan.c | 23 +++++------
src/backend/executor/nodeLimit.c | 1 +
src/backend/executor/nodeLockRows.c | 1 +
src/backend/executor/nodeMaterial.c | 5 ++-
src/backend/executor/nodeMemoize.c | 8 +++-
src/backend/executor/nodeMergeAppend.c | 3 ++
src/backend/executor/nodeMergejoin.c | 2 +
src/backend/executor/nodeModifyTable.c | 11 ++++-
src/backend/executor/nodeNestloop.c | 2 +
src/backend/executor/nodeProjectSet.c | 1 +
src/backend/executor/nodeRecursiveunion.c | 24 +++++++++--
src/backend/executor/nodeResult.c | 1 +
src/backend/executor/nodeSamplescan.c | 7 +++-
src/backend/executor/nodeSeqscan.c | 16 +++-----
src/backend/executor/nodeSetOp.c | 6 ++-
src/backend/executor/nodeSort.c | 5 ++-
src/backend/executor/nodeSubqueryscan.c | 1 +
src/backend/executor/nodeTableFuncscan.c | 4 +-
src/backend/executor/nodeTidrangescan.c | 12 ++++--
src/backend/executor/nodeTidscan.c | 8 +++-
src/backend/executor/nodeUnique.c | 1 +
src/backend/executor/nodeWindowAgg.c | 41 ++++++++++++++-----
38 files changed, 252 insertions(+), 121 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1393716587..802f76c73e 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2126,7 +2126,10 @@ postgresEndForeignModify(EState *estate,
{
PgFdwModifyState *fmstate = (PgFdwModifyState *) resultRelInfo->ri_FdwState;
- /* If fmstate is NULL, we are in EXPLAIN; nothing to do */
+ /*
+ * If fmstate is NULL, we are either in EXPLAIN or if BeginForeignModify
+ * wasn't called; nothing to do in any case.
+ */
if (fmstate == NULL)
return;
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..f7f18d3054 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -3010,6 +3010,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if no EvalPlanQualInit() was done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index f154f28902..af22b1676f 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4314,7 +4313,7 @@ ExecEndAgg(AggState *node)
* worker back into shared memory so that it can be picked up by the main
* process to report in EXPLAIN ANALYZE.
*/
- if (node->shared_info && IsParallelWorker())
+ if (node->shared_info != NULL && IsParallelWorker())
{
AggregateInstrumentation *si;
@@ -4327,10 +4326,16 @@ ExecEndAgg(AggState *node)
/* Make sure we have closed any open tuplesorts */
- if (node->sort_in)
+ if (node->sort_in != NULL)
+ {
tuplesort_end(node->sort_in);
- if (node->sort_out)
+ node->sort_in = NULL;
+ }
+ if (node->sort_out != NULL)
+ {
tuplesort_end(node->sort_out);
+ node->sort_out = NULL;
+ }
hashagg_reset_spill_state(node);
@@ -4346,19 +4351,25 @@ ExecEndAgg(AggState *node)
for (setno = 0; setno < numGroupingSets; setno++)
{
- if (pertrans->sortstates[setno])
+ if (pertrans->sortstates[setno] != NULL)
tuplesort_end(pertrans->sortstates[setno]);
}
}
/* And ensure any agg shutdown callbacks have been called */
for (setno = 0; setno < numGroupingSets; setno++)
+ {
ReScanExprContext(node->aggcontexts[setno]);
- if (node->hashcontext)
+ node->aggcontexts[setno] = NULL;
+ }
+ if (node->hashcontext != NULL)
+ {
ReScanExprContext(node->hashcontext);
+ node->hashcontext = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a2af221e05 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -399,7 +399,10 @@ ExecEndAppend(AppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(appendplans[i]);
+ appendplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..4abb0609a0 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -192,8 +192,8 @@ ExecEndBitmapAnd(BitmapAndState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2db0acfc76..d3f58c22f9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -648,40 +648,59 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
void
ExecEndBitmapHeapScan(BitmapHeapScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * extract information from the node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
+ if (node->tbmiterator != NULL)
+ {
tbm_end_iterate(node->tbmiterator);
- if (node->prefetch_iterator)
+ node->tbmiterator = NULL;
+ }
+ if (node->prefetch_iterator != NULL)
+ {
tbm_end_iterate(node->prefetch_iterator);
- if (node->tbm)
+ node->prefetch_iterator = NULL;
+ }
+ if (node->tbm != NULL)
+ {
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
+ node->tbm = NULL;
+ }
+ if (node->shared_tbmiterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_tbmiterator);
- if (node->shared_prefetch_iterator)
+ node->shared_tbmiterator = NULL;
+ }
+ if (node->shared_prefetch_iterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ node->shared_prefetch_iterator = NULL;
+ }
if (node->vmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->vmbuffer);
+ node->vmbuffer = InvalidBuffer;
+ }
if (node->pvmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->pvmbuffer);
+ node->pvmbuffer = InvalidBuffer;
+ }
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 7cf8532bc9..488f11a3ff 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -175,22 +175,21 @@ ExecReScanBitmapIndexScan(BitmapIndexScanState *node)
void
ExecEndBitmapIndexScan(BitmapIndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->biss_RelationDesc;
- indexScanDesc = node->biss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->biss_ScanDesc != NULL)
+ {
+ index_endscan(node->biss_ScanDesc);
+ node->biss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->biss_RelationDesc != NULL)
+ {
+ index_close(node->biss_RelationDesc, NoLock);
+ node->biss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..ace18593aa 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -210,8 +210,8 @@ ExecEndBitmapOr(BitmapOrState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 73913ebb18..3aba28285a 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -301,17 +301,20 @@ ExecEndForeignScan(ForeignScanState *node)
EState *estate = node->ss.ps.state;
/* Let the FDW shut down */
- if (plan->operation != CMD_SELECT)
+ if (node->fdwroutine != NULL)
{
- if (estate->es_epq_active == NULL)
- node->fdwroutine->EndDirectModify(node);
+ if (plan->operation != CMD_SELECT)
+ {
+ if (estate->es_epq_active == NULL)
+ node->fdwroutine->EndDirectModify(node);
+ }
+ else
+ node->fdwroutine->EndForeignScan(node);
}
- else
- node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index bb2500a469..1a3c8abdad 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -249,6 +249,7 @@ void
ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGather(node);
}
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 7a71a58509..c6fb45fee0 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -289,6 +289,7 @@ void
ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGatherMerge(node);
}
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 8c650f0e46..6dfe5a1d23 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,10 +226,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index e72f0986c2..88ba336882 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,13 +413,11 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* shut down the subplan
*/
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index aea44a9d56..6dc43b9ff2 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -861,7 +861,7 @@ ExecEndHashJoin(HashJoinState *node)
/*
* Free hash table
*/
- if (node->hj_HashTable)
+ if (node->hj_HashTable != NULL)
{
ExecHashTableDestroy(node->hj_HashTable);
node->hj_HashTable = NULL;
@@ -871,7 +871,9 @@ ExecEndHashJoin(HashJoinState *node)
* clean up subtrees
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index cd094a190c..28a0e81cb3 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,8 +1079,16 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ if (node->group_pivot != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->group_pivot);
+ node->group_pivot = NULL;
+ }
+ if (node->transfer_tuple != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ node->transfer_tuple = NULL;
+ }
/*
* Release tuplesort resources.
@@ -1100,6 +1108,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
* Shut down the subplan.
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f1db35665c..1f3843abe9 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -364,15 +364,6 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
void
ExecEndIndexOnlyScan(IndexOnlyScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->ioss_RelationDesc;
- indexScanDesc = node->ioss_ScanDesc;
-
/* Release VM buffer pin, if any. */
if (node->ioss_VMBuffer != InvalidBuffer)
{
@@ -380,13 +371,21 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
+ /* close the scan (no-op if we didn't start it) */
+ if (node->ioss_ScanDesc != NULL)
+ {
+ index_endscan(node->ioss_ScanDesc);
+ node->ioss_ScanDesc = NULL;
+ }
+
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->ioss_RelationDesc != NULL)
+ {
+ index_close(node->ioss_RelationDesc, NoLock);
+ node->ioss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 14b9c00217..32e1714f15 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -785,22 +785,21 @@ ExecIndexAdvanceArrayKeys(IndexArrayKeyInfo *arrayKeys, int numArrayKeys)
void
ExecEndIndexScan(IndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->iss_RelationDesc;
- indexScanDesc = node->iss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->iss_ScanDesc != NULL)
+ {
+ index_endscan(node->iss_ScanDesc);
+ node->iss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->iss_RelationDesc != NULL)
+ {
+ index_close(node->iss_RelationDesc, NoLock);
+ node->iss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 5654158e3e..a97bac9f6d 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,6 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..26fbe95c57 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -387,6 +387,7 @@ ExecEndLockRows(LockRowsState *node)
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 753ea28915..03c514900b 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -243,13 +243,16 @@ ExecEndMaterial(MaterialState *node)
* Release tuplestore resources
*/
if (node->tuplestorestate != NULL)
+ {
tuplestore_end(node->tuplestorestate);
- node->tuplestorestate = NULL;
+ node->tuplestorestate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 94bf479287..ee4749c852 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1043,6 +1043,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable != NULL)
{
int count;
uint64 mem = 0;
@@ -1089,12 +1090,17 @@ ExecEndMemoize(MemoizeState *node)
}
/* Remove the cache context */
- MemoryContextDelete(node->tableContext);
+ if (node->tableContext != NULL)
+ {
+ MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..0a42a04b19 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -333,7 +333,10 @@ ExecEndMergeAppend(MergeAppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(mergeplans[i]);
+ mergeplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 648fdd9a5f..4d7d73a684 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1646,7 +1646,9 @@ ExecEndMergeJoin(MergeJoinState *node)
* shut down the subplans
*/
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d21a178ad5..ea043c57c1 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4430,7 +4430,9 @@ ExecEndModifyTable(ModifyTableState *node)
for (j = 0; j < resultRelInfo->ri_NumSlotsInitialized; j++)
{
ExecDropSingleTupleTableSlot(resultRelInfo->ri_Slots[j]);
+ resultRelInfo->ri_Slots[j] = NULL;
ExecDropSingleTupleTableSlot(resultRelInfo->ri_PlanSlots[j]);
+ resultRelInfo->ri_PlanSlots[j] = NULL;
}
}
@@ -4438,12 +4440,16 @@ ExecEndModifyTable(ModifyTableState *node)
* Close all the partitioned tables, leaf partitions, and their indices
* and release the slot used for tuple routing, if set.
*/
- if (node->mt_partition_tuple_routing)
+ if (node->mt_partition_tuple_routing != NULL)
{
ExecCleanupTupleRouting(node, node->mt_partition_tuple_routing);
+ node->mt_partition_tuple_routing = NULL;
- if (node->mt_root_tuple_slot)
+ if (node->mt_root_tuple_slot != NULL)
+ {
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
+ node->mt_root_tuple_slot = NULL;
+ }
}
/*
@@ -4455,6 +4461,7 @@ ExecEndModifyTable(ModifyTableState *node)
* shut down subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index fc8f833d8b..76e02449f5 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -367,7 +367,9 @@ ExecEndNestLoop(NestLoopState *node)
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index b4bbdc89b1..e9b96416d3 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -324,6 +324,7 @@ ExecEndProjectSet(ProjectSetState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..f6d60bcd6c 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -272,20 +272,36 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table != NULL)
+ {
+ tuplestore_end(node->working_table);
+ node->working_table = NULL;
+ }
+ if (node->intermediate_table != NULL)
+ {
+ tuplestore_end(node->intermediate_table);
+ node->intermediate_table = NULL;
+ }
/* free subsidiary stuff including hashtable */
- if (node->tempContext)
+ if (node->tempContext != NULL)
+ {
MemoryContextDelete(node->tempContext);
- if (node->tableContext)
+ node->tempContext = NULL;
+ }
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index e9f5732f33..f15902e840 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -244,6 +244,7 @@ ExecEndResult(ResultState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 41c1ea37ad..a6813559e6 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -185,14 +185,17 @@ ExecEndSampleScan(SampleScanState *node)
/*
* Tell sampling function that we finished the scan.
*/
- if (node->tsmroutine->EndSampleScan)
+ if (node->tsmroutine != NULL && node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
if (node->ss.ss_currentScanDesc)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 49a5933aff..911266da07 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -183,18 +183,14 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
void
ExecEndSeqScan(SeqScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * get information from node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- if (scanDesc != NULL)
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 98c1b84d43..5c2861d243 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -583,10 +583,14 @@ void
ExecEndSetOp(SetOpState *node)
{
/* free subsidiary stuff including hashtable */
- if (node->tableContext)
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index eea7f2ae15..c8a35b64a8 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -307,13 +307,16 @@ ExecEndSort(SortState *node)
* Release tuplesort resources
*/
if (node->tuplesortstate != NULL)
+ {
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
- node->tuplesortstate = NULL;
+ node->tuplesortstate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 1ee6295660..91d7ae82ce 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -171,6 +171,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
* close down subquery
*/
ExecEndNode(node->subplan);
+ node->subplan = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index a60dcd4943..80ed4b26a8 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -217,8 +217,10 @@ ExecEndTableFuncScan(TableFuncScanState *node)
* Release tuplestore resources
*/
if (node->tupstore != NULL)
+ {
tuplestore_end(node->tupstore);
- node->tupstore = NULL;
+ node->tupstore = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index da622d3f5f..9147e4afa8 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -327,10 +327,14 @@ ExecReScanTidRangeScan(TidRangeScanState *node)
void
ExecEndTidRangeScan(TidRangeScanState *node)
{
- TableScanDesc scan = node->ss.ss_currentScanDesc;
-
- if (scan != NULL)
- table_endscan(scan);
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 15055077d0..74ec6afdcc 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -470,8 +470,14 @@ ExecReScanTidScan(TidScanState *node)
void
ExecEndTidScan(TidScanState *node)
{
- if (node->ss.ss_currentScanDesc)
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 01f951197c..13c556326a 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -169,6 +169,7 @@ void
ExecEndUnique(UniqueState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 77724a6daa..c4c6f009ba 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -1351,11 +1351,14 @@ release_partition(WindowAggState *winstate)
* any aggregate temp data). We don't rely on retail pfree because some
* aggregates might have allocated data we don't have direct pointers to.
*/
- MemoryContextResetAndDeleteChildren(winstate->partcontext);
- MemoryContextResetAndDeleteChildren(winstate->aggcontext);
+ if (winstate->partcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->partcontext);
+ if (winstate->aggcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->aggcontext);
for (i = 0; i < winstate->numaggs; i++)
{
- if (winstate->peragg[i].aggcontext != winstate->aggcontext)
+ if (winstate->peragg[i].aggcontext != NULL &&
+ winstate->peragg[i].aggcontext != winstate->aggcontext)
MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext);
}
@@ -2681,24 +2684,40 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
for (i = 0; i < node->numaggs; i++)
{
- if (node->peragg[i].aggcontext != node->aggcontext)
+ if (node->peragg[i].aggcontext != NULL &&
+ node->peragg[i].aggcontext != node->aggcontext)
MemoryContextDelete(node->peragg[i].aggcontext);
}
- MemoryContextDelete(node->partcontext);
- MemoryContextDelete(node->aggcontext);
+ if (node->partcontext != NULL)
+ {
+ MemoryContextDelete(node->partcontext);
+ node->partcontext = NULL;
+ }
+ if (node->aggcontext != NULL)
+ {
+ MemoryContextDelete(node->aggcontext);
+ node->aggcontext = NULL;
+ }
- pfree(node->perfunc);
- pfree(node->peragg);
+ if (node->perfunc != NULL)
+ {
+ pfree(node->perfunc);
+ node->perfunc = NULL;
+ }
+ if (node->peragg != NULL)
+ {
+ pfree(node->peragg);
+ node->peragg = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* -----------------
--
2.35.3
[application/octet-stream] v47-0004-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch (49.3K, 8-v47-0004-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch)
download | inline diff:
From 0fa7321f8c265515fefe53ef632ea09d64a87976 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:46 +0900
Subject: [PATCH v47 4/9] Adjustments to allow ExecutorStart() to sometimes
fail
Upon passing a plan tree from a CachedPlan to the executor, there's a
possibility that ExecutorStart() might return an incompletely set up
planstate tree. This can happen if the CachedPlan undergoes invalidation
during the ExecInitNode() initialization process. In such cases, the
execution should be reattempted using a fresh CachedPlan. Also, any
partially initialized EState must be cleaned up by invoking both
ExecutorEnd() and FreeExecutorState().
ExecutorStart() (and ExecutorStart_hook()) now return a Boolean telling
the caller if the plan initialization failed.
For the replan loop in that context, it makes more sense to have
ExecutorStart() either in the same scope or closer to where
GetCachedPlan() is invoked. So this commit modifies the following
sites:
* The ExecutorStart() call in ExplainOnePlan() is moved into a new
function ExplainQueryDesc() along with CreateQueryDesc(). Callers
of ExplainOnePlan() should now call the new function first.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and a new memory
context for those. One unintended consequence is that
CommandCounterIncrement() between queries in the PORTAL_MULTI_QUERY
case is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still works because the Snapshot registered
in QueryDesc/EState is updated to account for the CCI().
This commit also adds a new flag to EState called es_canceled that
complements es_finished to denote the new scenario where
ExecutorStart() returns with a partially setup planstate tree. Also,
to reset the AFTER trigger state that would have been set up in the
ExecutorStart(), this adds a new function AfterTriggerCancelQuery()
which is called from ExecutorEnd() (not ExecutorFinish()) when
es_canceled is true.
Note that this commit by itself doesn't make any functional change,
because the CachedPlan is not passed into the executor yet.
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 5 +-
src/backend/commands/createas.c | 9 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 6 +-
src/backend/commands/matview.c | 9 +-
src/backend/commands/portalcmds.c | 6 +-
src/backend/commands/prepare.c | 31 +-
src/backend/commands/trigger.c | 13 +
src/backend/executor/execMain.c | 44 ++-
src/backend/executor/execParallel.c | 6 +-
src/backend/executor/execUtils.c | 1 +
src/backend/executor/functions.c | 7 +-
src/backend/executor/spi.c | 48 ++-
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 346 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/commands/trigger.h | 1 +
src/include/executor/executor.h | 6 +-
src/include/nodes/execnodes.h | 3 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/portal.h | 2 +
24 files changed, 466 insertions(+), 282 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index a46f2db352..58cb62e872 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -330,7 +330,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -967,13 +967,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -996,6 +998,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 0e3547c35b..f7730c8702 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -568,8 +568,11 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 18b07c0200..4a950c03ff 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -329,8 +329,13 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 281c47b2ee..8d1fe5738b 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to have been invalidated after
+ * calling ExecutorStart().
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4873,6 +4905,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b287a2e84c..127d2a3b0a 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -802,7 +802,11 @@ execute_sql_string(const char *sql)
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 22b8b820c3..7083fb2350 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -412,8 +412,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..a1ee5c0acd 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,11 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here,
+ * because there's no CachedPlan.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..f8d0b0ee25 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal has a cached plan and
+ * it's found to be invalidated during the initialization of its plan
+ * trees, the plan must be regenerated.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +647,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 52177759ab..dd139432b9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -5009,6 +5009,19 @@ AfterTriggerBeginQuery(void)
afterTriggers.query_depth++;
}
+/* ----------
+ * AfterTriggerCancelQuery()
+ *
+ * Called from ExecutorEnd() if the query execution was canceled.
+ * ----------
+ */
+void
+AfterTriggerCancelQuery(void)
+{
+ /* Set to a value denoting that no query is active. */
+ afterTriggers.query_depth = -1;
+}
+
/* ----------
* AfterTriggerEndQuery()
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index de7bf7ca67..5755336abd 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,14 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- (void) InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
+
+ /* Mark execution as canceled if plan won't be executed. */
+ estate->es_canceled = !plan_valid;
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -325,6 +338,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
estate = queryDesc->estate;
Assert(estate != NULL);
+ Assert(!estate->es_canceled);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -429,7 +443,7 @@ standard_ExecutorFinish(QueryDesc *queryDesc)
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/* This should be run once and only once per Executor instance */
- Assert(!estate->es_finished);
+ Assert(!estate->es_finished && !estate->es_canceled);
/* Switch into per-query memory context */
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
@@ -488,11 +502,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
Assert(estate != NULL);
/*
- * Check that ExecutorFinish was called, unless in EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
- Assert(estate->es_finished ||
+ Assert(estate->es_finished || estate->es_canceled ||
(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -506,6 +520,14 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
UnregisterSnapshot(estate->es_snapshot);
UnregisterSnapshot(estate->es_crosscheck_snapshot);
+ /*
+ * Cancel trigger execution too if the query execution was canceled.
+ */
+ if (estate->es_canceled &&
+ !(estate->es_top_eflags &
+ (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
+ AfterTriggerCancelQuery();
+
/*
* Must switch out of context before destroying it
*/
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 457ee46faf..13d2820a41 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1437,7 +1437,11 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 16704c0c2f..f0f5740c26 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -151,6 +151,7 @@ CreateExecutorState(void)
estate->es_top_eflags = 0;
estate->es_instrument = 0;
estate->es_finished = false;
+ estate->es_canceled = false;
estate->es_exprcontexts = NIL;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 7e452ed743..606da72535 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -863,7 +863,12 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index f2cca807ef..814ff1390f 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2675,8 +2687,23 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2851,10 +2878,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2898,14 +2924,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..4f923bbcae 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1230,7 +1230,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1735,6 +1740,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2026,9 +2032,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 4ef349df8b..fcf9925ed4 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -118,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, NULL, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -428,19 +343,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -450,15 +367,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -474,6 +389,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -491,8 +408,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
NULL,
@@ -504,30 +421,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -539,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -584,7 +499,82 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ NULL,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -597,19 +587,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1196,7 +1187,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1217,9 +1208,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1236,33 +1228,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1271,26 +1256,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1345,12 +1343,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 430e3ca7dd..d4f7c29301 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -257,6 +257,7 @@ extern void ExecASTruncateTriggers(EState *estate,
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
+extern void AfterTriggerCancelQuery(void);
extern void AfterTriggerEndQuery(EState *estate);
extern void AfterTriggerFireDeferred(void);
extern void AfterTriggerEndXact(bool isCommit);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 72cbf120c5..10c5cda169 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 846eb32a1d..bb5734edb5 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -670,6 +670,9 @@ typedef struct EState
int es_top_eflags; /* eflags passed to ExecutorStart */
int es_instrument; /* OR of InstrumentOption flags */
bool es_finished; /* true when ExecutorFinish is done */
+ bool es_canceled; /* true when execution was canceled
+ * upon encountering that plan was invalided
+ * during ExecInitNode() */
List *es_exprcontexts; /* List of ExprContexts within EState */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v47-0003-Prepare-executor-to-support-detecting-CachedPlan.patch (41.3K, 9-v47-0003-Prepare-executor-to-support-detecting-CachedPlan.patch)
download | inline diff:
From 5062207c1e6879cf865508e1325dcade88d39cc6 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:12:04 +0900
Subject: [PATCH v47 3/9] Prepare executor to support detecting CachedPlan
invalidation
This adds checks at various points during the executor's
initialization of the plan tree to determine whether the originating
CachedPlan has become invalid as a result of taking locks on the
relations referenced in the plan. This includes addding the check
after every call to ExecOpenScanRelation() and to ExecInitNode(),
including the recursive ones to initialize child nodes.
If a given ExecInit*() function detects that the plan has become
invalid, it should return immediately even if the PlanState node
it's building may only be partially valid. That is crucial for
two reasons depending on where the check is:
* The checks following ExecOpenScanRelation() may find the plan
having become invalid because the requested relation was dropped
or had its schema changed concurrently in a manner that risks
unsafe operations in the code that follows. For example, it
might try to dereference a NULL pointer when the check failed
because the relation was dropped.
* For the checks following ExecInitNode(), the returned child
PlanState node might be only partially invalid. The code that
follows may misbehave if it depends on inspecting the child
PlanState. Note that this commit adds the check following all
calls of ExecInitNode() that exist in the code base, even at
sites where there is no code that might misbehave today, because
it might misbehave in the future. It seems like a good idea to
put the guards in place today rather than in the future when the
need arises.
To pass the CachedPlan that the executor will use for these checks,
this adds a new field to QueryDesc and a new parameter to
CreateQueryDesc(). No caller of CreateQueryDesc() is made to pass
an actual CachedPlan though, so there is no functional change.
Reviewed-by: Robert Haas
---
contrib/postgres_fdw/postgres_fdw.c | 4 +++
src/backend/commands/copyto.c | 3 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 2 +-
src/backend/commands/extension.c | 1 +
src/backend/commands/matview.c | 2 +-
src/backend/executor/execMain.c | 39 ++++++++++++++++++----
src/backend/executor/execParallel.c | 9 ++++-
src/backend/executor/execProcnode.c | 4 +++
src/backend/executor/functions.c | 1 +
src/backend/executor/nodeAgg.c | 2 ++
src/backend/executor/nodeAppend.c | 10 +++---
src/backend/executor/nodeBitmapAnd.c | 2 ++
src/backend/executor/nodeBitmapHeapscan.c | 4 +++
src/backend/executor/nodeBitmapOr.c | 2 ++
src/backend/executor/nodeCustom.c | 2 ++
src/backend/executor/nodeForeignscan.c | 4 +++
src/backend/executor/nodeGather.c | 2 ++
src/backend/executor/nodeGatherMerge.c | 2 ++
src/backend/executor/nodeGroup.c | 2 ++
src/backend/executor/nodeHash.c | 2 ++
src/backend/executor/nodeHashjoin.c | 4 +++
src/backend/executor/nodeIncrementalSort.c | 2 ++
src/backend/executor/nodeIndexonlyscan.c | 2 ++
src/backend/executor/nodeIndexscan.c | 2 ++
src/backend/executor/nodeLimit.c | 2 ++
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 2 ++
src/backend/executor/nodeMemoize.c | 2 ++
src/backend/executor/nodeMergeAppend.c | 4 ++-
src/backend/executor/nodeMergejoin.c | 4 +++
src/backend/executor/nodeModifyTable.c | 13 ++++++++
src/backend/executor/nodeNestloop.c | 4 +++
src/backend/executor/nodeProjectSet.c | 2 ++
src/backend/executor/nodeRecursiveunion.c | 4 +++
src/backend/executor/nodeResult.c | 2 ++
src/backend/executor/nodeSamplescan.c | 2 ++
src/backend/executor/nodeSeqscan.c | 2 ++
src/backend/executor/nodeSetOp.c | 2 ++
src/backend/executor/nodeSort.c | 2 ++
src/backend/executor/nodeSubqueryscan.c | 2 ++
src/backend/executor/nodeTidrangescan.c | 2 ++
src/backend/executor/nodeTidscan.c | 2 ++
src/backend/executor/nodeUnique.c | 2 ++
src/backend/executor/nodeWindowAgg.c | 2 ++
src/backend/executor/spi.c | 1 +
src/backend/tcop/pquery.c | 5 ++-
src/include/executor/execdesc.h | 4 +++
src/include/executor/executor.h | 10 ++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 ++++++++
51 files changed, 189 insertions(+), 17 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 802f76c73e..4fa6bb121c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2663,7 +2663,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index eaa3172793..0e3547c35b 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 13217807ee..281c47b2ee 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -572,7 +572,7 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
dest = None_Receiver;
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
+ queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..b287a2e84c 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,6 +797,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..22b8b820c3 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,7 +408,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index f7f18d3054..de7bf7ca67 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -263,7 +263,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ (void) InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
}
@@ -829,9 +829,13 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise. The latter case may occur if the CachedPlan that provides
+ * the plan tree (queryDesc->cplan) got invalidated during the initialization.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,11 +843,14 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
- TupleDesc tupType;
+ PlanState *planstate = NULL;
+ TupleDesc tupType = NULL;
ListCell *l;
int i;
+ Assert(queryDesc->planstate == NULL);
+ Assert(queryDesc->tupDesc == NULL);
+
/*
* Do permissions checks
*/
@@ -855,6 +862,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +894,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (unlikely(relation == NULL))
+ return false;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -956,6 +966,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
i++;
}
@@ -966,6 +978,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1009,6 +1023,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
+
+ return true;
}
/*
@@ -2858,7 +2874,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2947,6 +2964,13 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
subplanstate = ExecInitNode(subplan, rcestate, 0);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
+
+ /*
+ * All the necessary locks must already have been taken when
+ * initializing the parent's copy of subplanstate, so the CachedPlan,
+ * if any, should not have become invalid during ExecInitNode().
+ */
+ Assert(ExecPlanStillValid(rcestate));
}
/*
@@ -2988,6 +3012,9 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /* See the comment above. */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..457ee46faf 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Set up a QueryDesc for the query. While the leader might've sourced
+ * the plan tree from a CachedPlan, we don't have one here. This isn't
+ * an issue since the leader ensured the required locks, making our
+ * plan tree valid. Even as we get our own lock copies in
+ * ExecGetRangeTableRelation(), they're all already held by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 6098cdca69..f2264c7d84 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -136,6 +136,10 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * Callers should check upon returning that ExecPlanStillValid(estate)
+ * returns true before continuing further with its processing, because the
+ * returned PlanState might be only partially valid otherwise.
* ------------------------------------------------------------------------
*/
PlanState *
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..7e452ed743 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index af22b1676f..597d68139e 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return aggstate;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index a2af221e05..53ca9dc85d 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -185,8 +185,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ appendplanstates = (PlanState **) palloc0(nplans *
+ sizeof(PlanState *));
+ appendstate->appendplans = appendplanstates;
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +223,11 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
- appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4abb0609a0..7556be713c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmapandstate;
i++;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d3f58c22f9..f1f8e16b17 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -770,11 +770,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index ace18593aa..7d2bf45d9c 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmaporstate;
i++;
}
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index 28b5bb9353..a0befbd0c6 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return css;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 3aba28285a..336acff719 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 1a3c8abdad..c524022c04 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gatherstate;
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index c6fb45fee0..676faabef5 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 6dfe5a1d23..efa1c44ab4 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return grpstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 88ba336882..1a4bd5504e 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 6dc43b9ff2..c0919074b0 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 28a0e81cb3..621ffafe02 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return incrsortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 1f3843abe9..c555c14888 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -495,6 +495,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 32e1714f15..a3bd1f7fb0 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -908,6 +908,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a97bac9f6d..ab133f1580 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return limitstate;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 26fbe95c57..e1ef768571 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 03c514900b..c38eef099d 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index ee4749c852..a6bf66029c 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 0a42a04b19..52c3edf278 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -120,7 +120,7 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +151,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 4d7d73a684..634c6d9fe5 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ea043c57c1..95d909c1d0 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3985,6 +3985,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ /*
+ * ExecInitResultRelation() may have returned without initializing
+ * rootResultRelInfo if the plan got invalidated, so check.
+ */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4013,6 +4020,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ /* See the comment above. */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/*
* For child result relations, store the root result relation
* pointer. We do so for the convenience of places that want to
@@ -4039,6 +4050,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 76e02449f5..64a24cb965 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index e9b96416d3..706cc23a21 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return state;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index f6d60bcd6c..27dc318acb 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index f15902e840..6820d3bfd5 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return resstate;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index a6813559e6..02051fea51 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 911266da07..9e3ef94388 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 5c2861d243..475af4df24 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c8a35b64a8..9de717aa7c 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return sortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 91d7ae82ce..d9c10d1f6f 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 9147e4afa8..a7482aee50 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -378,6 +378,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidrangestate;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 74ec6afdcc..657411ef19 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -523,6 +523,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidstate;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 13c556326a..ee30688417 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index c4c6f009ba..1246d7919a 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2461,6 +2461,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..f2cca807ef 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2668,6 +2668,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ NULL,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..4ef349df8b 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -65,6 +65,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +78,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -145,7 +147,7 @@ ProcessQuery(PlannedStmt *plan,
/*
* Create the QueryDesc object
*/
- queryDesc = CreateQueryDesc(plan, sourceText,
+ queryDesc = CreateQueryDesc(plan, NULL, sourceText,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
@@ -493,6 +495,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ NULL,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index aeebe0e0ff..72cbf120c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,15 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the CachedPlan in es_cachedplan still valid?
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..846eb32a1d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one, or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..0a9e041d51 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Invoked by the executor for each relation lock acquired during the
+ * initialization of the plan tree within the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
[application/octet-stream] v47-0001-Remove-obsolete-executor-cleanup-code.patch (31.1K, 10-v47-0001-Remove-obsolete-executor-cleanup-code.patch)
download | inline diff:
From e015ba798b4b92bc5142ca94baaf12a1cda089d9 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:52:39 +0900
Subject: [PATCH v47 1/9] Remove obsolete executor cleanup code
This commit removes unnecessary ExecExprFreeContext() calls in ExecEnd*
routines as the actual cleanup is managed by FreeExecutorState. With
no remaining callers for ExecExprFreeContext(), this commit also
removes the function.
This commit also drops redundant ExecClearTuple() calls, as
ExecResetTupleTable() in ExecEndPlan() already takes care of resetting
all TupleTableSlots initialized with ExecInitScanTupleSlot() and
ExecInitExtraTupleSlot().
After these modifications, the ExecEnd*() routines for ValuesScan,
NamedTuplestoreScan, and WorkTableScan became redundant. Thus, this
commit removes them.
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execProcnode.c | 18 +++++--------
src/backend/executor/execUtils.c | 26 -------------------
src/backend/executor/nodeAgg.c | 10 -------
src/backend/executor/nodeBitmapHeapscan.c | 12 ---------
src/backend/executor/nodeBitmapIndexscan.c | 8 ------
src/backend/executor/nodeCtescan.c | 12 ---------
src/backend/executor/nodeCustom.c | 7 -----
src/backend/executor/nodeForeignscan.c | 8 ------
src/backend/executor/nodeFunctionscan.c | 15 -----------
src/backend/executor/nodeGather.c | 3 ---
src/backend/executor/nodeGatherMerge.c | 3 ---
src/backend/executor/nodeGroup.c | 5 ----
src/backend/executor/nodeHash.c | 5 ----
src/backend/executor/nodeHashjoin.c | 12 ---------
src/backend/executor/nodeIncrementalSort.c | 5 ----
src/backend/executor/nodeIndexonlyscan.c | 16 ------------
src/backend/executor/nodeIndexscan.c | 16 ------------
src/backend/executor/nodeLimit.c | 1 -
src/backend/executor/nodeMaterial.c | 5 ----
src/backend/executor/nodeMemoize.c | 9 -------
src/backend/executor/nodeMergejoin.c | 12 ---------
src/backend/executor/nodeModifyTable.c | 11 --------
.../executor/nodeNamedtuplestorescan.c | 22 ----------------
src/backend/executor/nodeNestloop.c | 11 --------
src/backend/executor/nodeProjectSet.c | 10 -------
src/backend/executor/nodeResult.c | 10 -------
src/backend/executor/nodeSamplescan.c | 12 ---------
src/backend/executor/nodeSeqscan.c | 12 ---------
src/backend/executor/nodeSetOp.c | 4 ---
src/backend/executor/nodeSort.c | 7 -----
src/backend/executor/nodeSubqueryscan.c | 12 ---------
src/backend/executor/nodeTableFuncscan.c | 12 ---------
src/backend/executor/nodeTidrangescan.c | 12 ---------
src/backend/executor/nodeTidscan.c | 12 ---------
src/backend/executor/nodeUnique.c | 5 ----
src/backend/executor/nodeValuesscan.c | 24 -----------------
src/backend/executor/nodeWindowAgg.c | 17 ------------
src/backend/executor/nodeWorktablescan.c | 22 ----------------
src/include/executor/executor.h | 1 -
.../executor/nodeNamedtuplestorescan.h | 1 -
src/include/executor/nodeValuesscan.h | 1 -
src/include/executor/nodeWorktablescan.h | 1 -
42 files changed, 6 insertions(+), 421 deletions(-)
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..6098cdca69 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -667,22 +667,10 @@ ExecEndNode(PlanState *node)
ExecEndTableFuncScan((TableFuncScanState *) node);
break;
- case T_ValuesScanState:
- ExecEndValuesScan((ValuesScanState *) node);
- break;
-
case T_CteScanState:
ExecEndCteScan((CteScanState *) node);
break;
- case T_NamedTuplestoreScanState:
- ExecEndNamedTuplestoreScan((NamedTuplestoreScanState *) node);
- break;
-
- case T_WorkTableScanState:
- ExecEndWorkTableScan((WorkTableScanState *) node);
- break;
-
case T_ForeignScanState:
ExecEndForeignScan((ForeignScanState *) node);
break;
@@ -757,6 +745,12 @@ ExecEndNode(PlanState *node)
ExecEndLimit((LimitState *) node);
break;
+ /* No clean up actions for these nodes. */
+ case T_ValuesScanState:
+ case T_NamedTuplestoreScanState:
+ case T_WorkTableScanState:
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(node));
break;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..16704c0c2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -638,32 +638,6 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, int varno, TupleDesc tupdesc)
return true;
}
-/* ----------------
- * ExecFreeExprContext
- *
- * A plan node's ExprContext should be freed explicitly during executor
- * shutdown because there may be shutdown callbacks to call. (Other resources
- * made by the above routines, such as projection info, don't need to be freed
- * explicitly because they're just memory in the per-query memory context.)
- *
- * However ... there is no particular need to do it during ExecEndNode,
- * because FreeExecutorState will free any remaining ExprContexts within
- * the EState. Letting FreeExecutorState do it allows the ExprContexts to
- * be freed in reverse order of creation, rather than order of creation as
- * will happen if we delete them here, which saves O(N^2) work in the list
- * cleanup inside FreeExprContext.
- * ----------------
- */
-void
-ExecFreeExprContext(PlanState *planstate)
-{
- /*
- * Per above discussion, don't actually delete the ExprContext. We do
- * unlink it from the plan node, though.
- */
- planstate->ps_ExprContext = NULL;
-}
-
/* ----------------------------------------------------------------
* Scan node support
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..f154f28902 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4357,16 +4357,6 @@ ExecEndAgg(AggState *node)
if (node->hashcontext)
ReScanExprContext(node->hashcontext);
- /*
- * We don't actually free any ExprContexts here (see comment in
- * ExecFreeExprContext), just unlinking the output one from the plan node
- * suffices.
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..2db0acfc76 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -655,18 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..7cf8532bc9 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -184,14 +184,6 @@ ExecEndBitmapIndexScan(BitmapIndexScanState *node)
indexRelationDesc = node->biss_RelationDesc;
indexScanDesc = node->biss_ScanDesc;
- /*
- * Free the exprcontext ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- if (node->biss_RuntimeContext)
- FreeExprContext(node->biss_RuntimeContext, true);
-#endif
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index cc4c4243e2..a0c0c4be33 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -287,18 +287,6 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
void
ExecEndCteScan(CteScanState *node)
{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* If I am the leader, free the tuplestore.
*/
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..28b5bb9353 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -129,13 +129,6 @@ ExecEndCustomScan(CustomScanState *node)
{
Assert(node->methods->EndCustomScan != NULL);
node->methods->EndCustomScan(node);
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* Clean out the tuple table */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..73913ebb18 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -312,14 +312,6 @@ ExecEndForeignScan(ForeignScanState *node)
/* Shut down any outer plan. */
if (outerPlanState(node))
ExecEndNode(outerPlanState(node));
-
- /* Free the exprcontext */
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean out the tuple table */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index dd06ef8aee..2dddbcda14 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -523,18 +523,6 @@ ExecEndFunctionScan(FunctionScanState *node)
{
int i;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release slots and tuplestore resources
*/
@@ -542,9 +530,6 @@ ExecEndFunctionScan(FunctionScanState *node)
{
FunctionScanPerFuncState *fs = &node->funcstates[i];
- if (fs->func_slot)
- ExecClearTuple(fs->func_slot);
-
if (fs->tstore != NULL)
{
tuplestore_end(node->funcstates[i].tstore);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..bb2500a469 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -250,9 +250,6 @@ ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGather(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..7a71a58509 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -290,9 +290,6 @@ ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
ExecShutdownGatherMerge(node);
- ExecFreeExprContext(&node->ps);
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..8c650f0e46 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -228,11 +228,6 @@ ExecEndGroup(GroupState *node)
{
PlanState *outerPlan;
- ExecFreeExprContext(&node->ss.ps);
-
- /* clean up tuple table */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
outerPlan = outerPlanState(node);
ExecEndNode(outerPlan);
}
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..e72f0986c2 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -415,11 +415,6 @@ ExecEndHash(HashState *node)
{
PlanState *outerPlan;
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..aea44a9d56 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -867,18 +867,6 @@ ExecEndHashJoin(HashJoinState *node)
node->hj_HashTable = NULL;
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->hj_OuterTupleSlot);
- ExecClearTuple(node->hj_HashTupleSlot);
-
/*
* clean up subtrees
*/
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..cd094a190c 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,11 +1079,6 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- /* clean out the scan tuple */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- /* must drop standalone tuple slots from outer node */
ExecDropSingleTupleTableSlot(node->group_pivot);
ExecDropSingleTupleTableSlot(node->transfer_tuple);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..f1db35665c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -380,22 +380,6 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->ioss_RuntimeContext)
- FreeExprContext(node->ioss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..14b9c00217 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -794,22 +794,6 @@ ExecEndIndexScan(IndexScanState *node)
indexRelationDesc = node->iss_RelationDesc;
indexScanDesc = node->iss_ScanDesc;
- /*
- * Free the exprcontext(s) ... now dead code, see ExecFreeExprContext
- */
-#ifdef NOT_USED
- ExecFreeExprContext(&node->ss.ps);
- if (node->iss_RuntimeContext)
- FreeExprContext(node->iss_RuntimeContext, true);
-#endif
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close the index relation (no-op if we didn't open it)
*/
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..5654158e3e 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -534,7 +534,6 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
void
ExecEndLimit(LimitState *node)
{
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..753ea28915 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -239,11 +239,6 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
void
ExecEndMaterial(MaterialState *node)
{
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..94bf479287 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1091,15 +1091,6 @@ ExecEndMemoize(MemoizeState *node)
/* Remove the cache context */
MemoryContextDelete(node->tableContext);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to cache result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
- /*
- * free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
/*
* shut down the subplan
*/
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..648fdd9a5f 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1642,18 +1642,6 @@ ExecEndMergeJoin(MergeJoinState *node)
{
MJ1_printf("ExecEndMergeJoin: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->mj_MarkedTupleSlot);
-
/*
* shut down the subplans
*/
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5005d8c0d1..d21a178ad5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4446,17 +4446,6 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* Terminate EPQ execution if active
*/
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index 46832ad82f..3547dc2b10 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -155,28 +155,6 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndNamedTuplestoreScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanNamedTuplestoreScan
*
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..fc8f833d8b 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -363,17 +363,6 @@ ExecEndNestLoop(NestLoopState *node)
{
NL1_printf("ExecEndNestLoop: %s\n",
"ending node processing");
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->js.ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
-
/*
* close down subplans
*/
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..b4bbdc89b1 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -320,16 +320,6 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
void
ExecEndProjectSet(ProjectSetState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..e9f5732f33 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -240,16 +240,6 @@ ExecInitResult(Result *node, EState *estate, int eflags)
void
ExecEndResult(ResultState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ps);
-
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/*
* shut down subplans
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..41c1ea37ad 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -188,18 +188,6 @@ ExecEndSampleScan(SampleScanState *node)
if (node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..49a5933aff 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -190,18 +190,6 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close heap scan
*/
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..98c1b84d43 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -582,13 +582,9 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
void
ExecEndSetOp(SetOpState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
/* free subsidiary stuff including hashtable */
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..eea7f2ae15 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -303,13 +303,6 @@ ExecEndSort(SortState *node)
SO1_printf("ExecEndSort: %s\n",
"shutting down sort node");
- /*
- * clean out the tuple table
- */
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /* must drop pointer to sort result tuple */
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
-
/*
* Release tuplesort resources
*/
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..1ee6295660 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -167,18 +167,6 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
void
ExecEndSubqueryScan(SubqueryScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the upper tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* close down subquery
*/
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 791cbd2372..a60dcd4943 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -213,18 +213,6 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
void
ExecEndTableFuncScan(TableFuncScanState *node)
{
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-
/*
* Release tuplestore resources
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..da622d3f5f 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -331,18 +331,6 @@ ExecEndTidRangeScan(TidRangeScanState *node)
if (scan != NULL)
table_endscan(scan);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..15055077d0 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -472,18 +472,6 @@ ExecEndTidScan(TidScanState *node)
{
if (node->ss.ss_currentScanDesc)
table_endscan(node->ss.ss_currentScanDesc);
-
- /*
- * Free the exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clear out tuple table slots
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..01f951197c 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -168,11 +168,6 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
void
ExecEndUnique(UniqueState *node)
{
- /* clean up tuple table */
- ExecClearTuple(node->ps.ps_ResultTupleSlot);
-
- ExecFreeExprContext(&node->ps);
-
ExecEndNode(outerPlanState(node));
}
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index 32ace63017..fbfb067f3b 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -319,30 +319,6 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndValuesScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndValuesScan(ValuesScanState *node)
-{
- /*
- * Free both exprcontexts
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->rowcontext;
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanValuesScan
*
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..77724a6daa 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2686,23 +2686,6 @@ ExecEndWindowAgg(WindowAggState *node)
release_partition(node);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
- ExecClearTuple(node->first_part_slot);
- ExecClearTuple(node->agg_row_slot);
- ExecClearTuple(node->temp_slot_1);
- ExecClearTuple(node->temp_slot_2);
- if (node->framehead_slot)
- ExecClearTuple(node->framehead_slot);
- if (node->frametail_slot)
- ExecClearTuple(node->frametail_slot);
-
- /*
- * Free both the expr contexts.
- */
- ExecFreeExprContext(&node->ss.ps);
- node->ss.ps.ps_ExprContext = node->tmpcontext;
- ExecFreeExprContext(&node->ss.ps);
-
for (i = 0; i < node->numaggs; i++)
{
if (node->peragg[i].aggcontext != node->aggcontext)
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 0c13448236..17a548865e 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -181,28 +181,6 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
return scanstate;
}
-/* ----------------------------------------------------------------
- * ExecEndWorkTableScan
- *
- * frees any storage allocated through C routines.
- * ----------------------------------------------------------------
- */
-void
-ExecEndWorkTableScan(WorkTableScanState *node)
-{
- /*
- * Free exprcontext
- */
- ExecFreeExprContext(&node->ss.ps);
-
- /*
- * clean out the tuple table
- */
- if (node->ss.ps.ps_ResultTupleSlot)
- ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
- ExecClearTuple(node->ss.ss_ScanTupleSlot);
-}
-
/* ----------------------------------------------------------------
* ExecReScanWorkTableScan
*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..aeebe0e0ff 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -569,7 +569,6 @@ extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
-extern void ExecFreeExprContext(PlanState *planstate);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/nodeNamedtuplestorescan.h b/src/include/executor/nodeNamedtuplestorescan.h
index 3ff687023a..9d80236fe5 100644
--- a/src/include/executor/nodeNamedtuplestorescan.h
+++ b/src/include/executor/nodeNamedtuplestorescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern NamedTuplestoreScanState *ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflags);
-extern void ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node);
extern void ExecReScanNamedTuplestoreScan(NamedTuplestoreScanState *node);
#endif /* NODENAMEDTUPLESTORESCAN_H */
diff --git a/src/include/executor/nodeValuesscan.h b/src/include/executor/nodeValuesscan.h
index a52fa678df..fe3f043951 100644
--- a/src/include/executor/nodeValuesscan.h
+++ b/src/include/executor/nodeValuesscan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern ValuesScanState *ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags);
-extern void ExecEndValuesScan(ValuesScanState *node);
extern void ExecReScanValuesScan(ValuesScanState *node);
#endif /* NODEVALUESSCAN_H */
diff --git a/src/include/executor/nodeWorktablescan.h b/src/include/executor/nodeWorktablescan.h
index e553a453f3..f31b22cec4 100644
--- a/src/include/executor/nodeWorktablescan.h
+++ b/src/include/executor/nodeWorktablescan.h
@@ -17,7 +17,6 @@
#include "nodes/execnodes.h"
extern WorkTableScanState *ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags);
-extern void ExecEndWorkTableScan(WorkTableScanState *node);
extern void ExecReScanWorkTableScan(WorkTableScanState *node);
#endif /* NODEWORKTABLESCAN_H */
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-09-28 08:26 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Langote @ 2023-09-28 08:26 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Tue, Sep 26, 2023 at 10:06 PM Amit Langote <[email protected]> wrote:
> On Mon, Sep 25, 2023 at 9:57 PM Amit Langote <[email protected]> wrote:
> > On Wed, Sep 6, 2023 at 11:20 PM Robert Haas <[email protected]> wrote:
> > > - Is there any point to all of these early exit cases? For example, in
> > > ExecInitBitmapAnd, why exit early if initialization fails? Why not
> > > just plunge ahead and if initialization failed the caller will notice
> > > that and when we ExecEndNode some of the child node pointers will be
> > > NULL but who cares? The obvious disadvantage of this approach is that
> > > we're doing a bunch of unnecessary initialization, but we're also
> > > speeding up the common case where we don't need to abort by avoiding a
> > > branch that will rarely be taken. I'm not quite sure what the right
> > > thing to do is here.
> > I thought about this some and figured that adding the
> > is-CachedPlan-still-valid tests in the following places should suffice
> > after all:
> >
> > 1. In InitPlan() right after the top-level ExecInitNode() calls
> > 2. In ExecInit*() functions of Scan nodes, right after
> > ExecOpenScanRelation() calls
>
> After sleeping on this, I think we do need the checks after all the
> ExecInitNode() calls too, because we have many instances of the code
> like the following one:
>
> outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
> tupDesc = ExecGetResultType(outerPlanState(gatherstate));
> <some code that dereferences outDesc>
>
> If outerNode is a SeqScan and ExecInitSeqScan() returned early because
> ExecOpenScanRelation() detected that plan was invalidated, then
> tupDesc would be NULL in this case, causing the code to crash.
>
> Now one might say that perhaps we should only add the
> is-CachedPlan-valid test in the instances where there is an actual
> risk of such misbehavior, but that could lead to confusion, now or
> later. It seems better to add them after every ExecInitNode() call
> while we're inventing the notion, because doing so relieves the
> authors of future enhancements of the ExecInit*() routines from
> worrying about any of this.
>
> Attached 0003 should show how that turned out.
>
> Updated 0002 as mentioned in the previous reply -- setting pointers to
> NULL after freeing them more consistently across various ExecEnd*()
> routines and using the `if (pointer != NULL)` style over the `if
> (pointer)` more consistently.
>
> Updated 0001's commit message to remove the mention of its relation to
> any future commits. I intend to push it tomorrow.
Pushed that one. Here are the rebased patches.
0001 seems ready to me, but I'll wait a couple more days for others to
weigh in. Just to highlight a kind of change that others may have
differing opinions on, consider this hunk from the patch:
- MemoryContextDelete(node->aggcontext);
+ if (node->aggcontext != NULL)
+ {
+ MemoryContextDelete(node->aggcontext);
+ node->aggcontext = NULL;
+ }
...
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
So the patch wants to enhance the consistency of setting the pointer
to NULL after freeing part. Robert mentioned his preference for doing
it in the patch, which I agree with.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v48-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch (25.2K, 2-v48-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From 8c0ff924890d173ddd9c6c087a725aeaccd210a0 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:15 +0900
Subject: [PATCH v48 7/8] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
Previous commits have made all the necessary adjustment to make the
executor lock child tables, to detect invalidation of the CachedPlan
resulting from that, and to retry the execution with a new CachePlan.
So, this commit simply removes the code in plancache.c that does the
"for execution" locking, aka AcquireExecutorLocks().
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/spi.c | 2 +-
src/backend/tcop/pquery.c | 6 +-
src/backend/utils/cache/plancache.c | 154 +++++++----------
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 158 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
7 files changed, 343 insertions(+), 108 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 814ff1390f..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2680,7 +2680,7 @@ replan:
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
- NULL,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index fcf9925ed4..8d0772ae29 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -412,7 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
- NULL,
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -443,6 +443,7 @@ PortalStart(Portal portal, ParamListInfo params,
*/
if (!ExecutorStart(queryDesc, myeflags))
{
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
PopActiveSnapshot();
@@ -542,7 +543,7 @@ PortalStart(Portal portal, ParamListInfo params,
* PortalRunMulti() before calling ExecutorRun().
*/
queryDesc = CreateQueryDesc(plan,
- NULL,
+ portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -566,6 +567,7 @@ PortalStart(Portal portal, ParamListInfo params,
if (!ExecutorStart(queryDesc, myeflags))
{
PopActiveSnapshot();
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
plan_valid = false;
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 7d4168f82f..35d903cb98 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -104,13 +104,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -792,8 +792,13 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan includes child relations introduced by the planner, they
+ * wouldn't be locked yet. This is because AcquirePlannerLocks() only locks
+ * relations present in the original query's range table (before planner
+ * entry). Hence, the plan might become stale if child relations are modified
+ * concurrently. During the plan initialization, the executor must ensure the
+ * plan (CachedPlan) remains valid after locking each child table. If found
+ * invalid, the caller should be prompted to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -807,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1130,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * Typically, the plan returned by this function is valid. However, a caveat
+ * arises with inheritance/partition child tables. These aren't locked by
+ * this function, as we only lock tables directly mentioned in the original
+ * query here. The task of locking these child tables falls to the executor
+ * during plan tree setup. If acquiring these locks invalidates the plan, the
+ * executor should inform the caller to regenerate the plan by invoking this
+ * function again. The reason for this deferred child table locking mechanism
+ * is efficiency: not all might need to be locked. Some could be pruned during
+ * executor initialization, especially if their corresponding plan nodes
+ * facilitate partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1166,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1364,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1741,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..122d81f2ee
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,158 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+----------------------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = $1)
+(7 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(5 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..2d0607b176
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo11 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v48-0006-Add-field-to-store-parent-relids-to-Append-Merge.patch (26.1K, 3-v48-0006-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From bf5f136b6c6eaf278f2aefde525074afbef3958f Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:02 +0900
Subject: [PATCH v48 6/8] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo. A future commit will create a need for them to
be available at all times for the purpose of locking those
parent/ancestor tables when executing a cached plan, so add a
field called allpartrelids to Append/MergeAppend to store those
RT indexes. This also adds a function called
ExecLockAppendNonLeafTables() to lock those tables.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 2 +-
src/backend/executor/execUtils.c | 33 ++++++
src/backend/executor/nodeAppend.c | 14 +++
src/backend/executor/nodeMergeAppend.c | 14 +++
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/executor/executor.h | 1 +
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
13 files changed, 266 insertions(+), 124 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ffc62e379a..2804ec70f1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1475,7 +1475,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafPartitions().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 117773706a..2b7a08c9ba 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -827,6 +827,39 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafPartitions
+ * Lock non-leaf partitions whose child partitions are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i = -1;
+
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ /*
+ * Don't lock the root parent mentioned in the query, because it
+ * should already have been locked before entering the executor.
+ */
+ if (!rte->inFromCl)
+ LockRelationOid(rte->relid, rte->rellockmode);
+ else
+ Assert(CheckRelLockedByMe(rte->relid, rte->rellockmode, true));
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 53ca9dc85d..4759511f87 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,20 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 52c3edf278..158210aac1 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,20 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..d1f4f606bf 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1229,6 +1230,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1370,15 +1372,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1399,7 +1409,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1445,6 +1456,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1534,15 +1546,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1554,7 +1574,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 7962200885..8e256dd779 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 10c5cda169..74a471e3e3 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -601,6 +601,7 @@ exec_rt_fetch(Index rti, EState *estate)
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
+extern void ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids);
extern int executor_errposition(EState *estate, int location);
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v48-0008-Track-opened-range-table-relations-in-a-List-in-.patch (2.5K, 4-v48-0008-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From fef4457e294bcc6b48a910f148816b2d163905ec Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:19 +0900
Subject: [PATCH v48 8/8] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 3 +++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 2804ec70f1..d559c1de61 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1649,12 +1649,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 2b7a08c9ba..1dfef44495 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -822,6 +822,9 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ if (rel != NULL)
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index bb5734edb5..8bbe1f6b14 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v48-0005-Assert-that-relations-needing-their-permissions-.patch (5.1K, 5-v48-0005-Assert-that-relations-needing-their-permissions-.patch)
download | inline diff:
From ff1dfbb0df6d86acb5d4d6dabee623d74df17ab7 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Mon, 25 Sep 2023 11:52:02 +0900
Subject: [PATCH v48 5/8] Assert that relations needing their permissions
checked are locked
---
src/backend/executor/execMain.c | 11 +++++++
src/backend/storage/lmgr/lmgr.c | 45 +++++++++++++++++++++++++++++
src/backend/utils/cache/lsyscache.c | 21 ++++++++++++++
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
5 files changed, 79 insertions(+)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5755336abd..ffc62e379a 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -626,6 +626,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
(rte->rtekind == RTE_SUBQUERY &&
rte->relkind == RELKIND_VIEW));
+ /*
+ * Relations whose permissions need to be checked must already
+ * have been locked by the parser or by GetCachedPlan() if a
+ * cached plan is being executed.
+ *
+ * XXX Maybe we should we skip calling ExecCheckPermissions from
+ * InitPlan in a parallel worker.
+ */
+ Assert(IsParallelWorker() ||
+ CheckRelLockedByMe(rte->relid, AccessShareLock, true));
+
(void) getRTEPermissionInfo(rteperminfos, rte);
/* Many-to-one mapping not allowed */
Assert(!bms_is_member(rte->perminfoindex, indexset));
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
--
2.35.3
[application/octet-stream] v48-0004-Teach-the-executor-to-lock-child-tables-in-some-.patch (11.1K, 6-v48-0004-Teach-the-executor-to-lock-child-tables-in-some-.patch)
download | inline diff:
From 49787b3c0ed2a14ca5a68f33b47e892cf90b913f Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:17:15 +0900
Subject: [PATCH v48 4/8] Teach the executor to lock child tables in some cases
An upcoming commit will move the locking of child tables referenced
in a cached plan tree from GetCachedPlan() to the executor
initialization of the plan tree in ExecutorStart(). This commit
teaches ExecGetRangeTableRelation() to lock child tables if
EState.es_cachedplan points to a CachedPlan.
The executor must now deal with the cases where an unlocked child
table might have been concurrently dropped, so this modifies
ExecGetRangeTableRelation() to use try_table_open(). All of its
callers (and those of ExecOpenScanRelation() that calls it) must
now account for the child table disappearing, which means to abort
initializing the table's Scan node in the middle.
ExecGetRangeTableRelation() now examines inFromCl field of an RTE
to determine that a given range table relation is a child table, so
this commit also makes the planner set inFromCl to false in the
child tables' RTEs that it manufactures.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/executor/README | 36 +++++++++++++++++++++++-
src/backend/executor/execPartition.c | 2 ++
src/backend/executor/execUtils.c | 41 +++++++++++++++++++++-------
src/backend/optimizer/util/inherit.c | 7 +++++
src/backend/parser/analyze.c | 7 ++---
src/include/nodes/parsenodes.h | 8 ++++--
6 files changed, 84 insertions(+), 17 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..6d2240610d 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,34 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Typically, when the executor initializes a plan tree for execution, it doesn't
+lock non-index relations if the plan tree is freshly generated and not derived
+from a CachedPlan. This is because such locks have already been established
+during the query's parsing, rewriting, and planning phases. However, with a
+cached plan tree, there can be relations that remain unlocked. The function
+GetCachedPlan() locks relations existing in the query's range table pre-planning
+but doesn't account for those added during the planning phase. Consequently,
+inheritance child tables, introduced to the query's range table during planning,
+won't be locked when the cached plan reaches the executor.
+
+The decision to defer locking child tables with GetCachedPlan() arises from the
+fact that not all might be accessed during plan execution. For instance, if
+child tables are partitions, some might be omitted due to pruning at
+execution-initialization-time. Thus, the responsibility of locking these child
+tables is pushed to execution-initialization-time, taking place in ExecInitNode()
+for plan nodes encompassing these tables.
+
+This approach opens a window where a cached plan tree with child tables could
+become outdated if another backend modifies these tables before ExecInitNode()
+locks them. Given this, the executor has the added duty to confirm the plan
+tree's validity whenever it locks a child table post execution-initialization-
+pruning. This validation is done by checking the CachedPlan.is_valid attribute
+of the CachedPlan provided. If the plan tree is outdated (is_valid=false), the
+executor halts any further initialization and alerts the caller that they should
+retry execution with another freshly created plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +344,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..84978c5525 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1927,6 +1927,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (unlikely(partrel == NULL))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f0f5740c26..117773706a 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -697,6 +697,8 @@ ExecRelationIsTargetRelation(EState *estate, Index scanrelid)
*
* Open the heap relation to be scanned by a base-level scan plan node.
* This should be called during the node's ExecInit routine.
+ *
+ * NULL is returned if the relation is found to have been dropped.
* ----------------------------------------------------------------
*/
Relation
@@ -706,6 +708,8 @@ ExecOpenScanRelation(EState *estate, Index scanrelid, int eflags)
/* Open the relation. */
rel = ExecGetRangeTableRelation(estate, scanrelid);
+ if (unlikely(rel == NULL))
+ return NULL;
/*
* Complain if we're attempting a scan of an unscannable relation, except
@@ -763,6 +767,9 @@ ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos)
* Open the Relation for a range table entry, if not already done
*
* The Relations will be closed again in ExecEndPlan().
+ *
+ * Returned value may be NULL if the relation is a child relation that is not
+ * already locked.
*/
Relation
ExecGetRangeTableRelation(EState *estate, Index rti)
@@ -779,7 +786,28 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ *
+ * Note that we use try_table_open() here, because without a lock
+ * held on the relation, it may have disappeared from under us.
+ */
+ rel = try_table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -792,15 +820,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -823,6 +842,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (unlikely(resultRelationDesc == NULL))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..1b30c0ff87 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,13 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+
+ /*
+ * Flag child tables as indirectly referenced in the query. This helps
+ * the executor's ExecGetRangeTableRelation() recognize them as
+ * inheritance children.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 7a1dfb6364..cf269f8c53 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3305,10 +3305,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f637937cd2..acf87580a1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,15 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. The executor uses
+ * this to ascertain if an RTE_RELATION entry is for a table explicitly
+ * named in the query or a child table added by the planner. This
+ * distinction is vital when child tables in a plan must be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v48-0003-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch (49.3K, 7-v48-0003-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch)
download | inline diff:
From 36eb6e04907d4ab44ad424e11d54f9589309d0d2 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:46 +0900
Subject: [PATCH v48 3/8] Adjustments to allow ExecutorStart() to sometimes
fail
Upon passing a plan tree from a CachedPlan to the executor, there's a
possibility that ExecutorStart() might return an incompletely set up
planstate tree. This can happen if the CachedPlan undergoes invalidation
during the ExecInitNode() initialization process. In such cases, the
execution should be reattempted using a fresh CachedPlan. Also, any
partially initialized EState must be cleaned up by invoking both
ExecutorEnd() and FreeExecutorState().
ExecutorStart() (and ExecutorStart_hook()) now return a Boolean telling
the caller if the plan initialization failed.
For the replan loop in that context, it makes more sense to have
ExecutorStart() either in the same scope or closer to where
GetCachedPlan() is invoked. So this commit modifies the following
sites:
* The ExecutorStart() call in ExplainOnePlan() is moved into a new
function ExplainQueryDesc() along with CreateQueryDesc(). Callers
of ExplainOnePlan() should now call the new function first.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and a new memory
context for those. One unintended consequence is that
CommandCounterIncrement() between queries in the PORTAL_MULTI_QUERY
case is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still works because the Snapshot registered
in QueryDesc/EState is updated to account for the CCI().
This commit also adds a new flag to EState called es_canceled that
complements es_finished to denote the new scenario where
ExecutorStart() returns with a partially setup planstate tree. Also,
to reset the AFTER trigger state that would have been set up in the
ExecutorStart(), this adds a new function AfterTriggerCancelQuery()
which is called from ExecutorEnd() (not ExecutorFinish()) when
es_canceled is true.
Note that this commit by itself doesn't make any functional change,
because the CachedPlan is not passed into the executor yet.
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 5 +-
src/backend/commands/createas.c | 9 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 6 +-
src/backend/commands/matview.c | 9 +-
src/backend/commands/portalcmds.c | 6 +-
src/backend/commands/prepare.c | 31 +-
src/backend/commands/trigger.c | 13 +
src/backend/executor/execMain.c | 44 ++-
src/backend/executor/execParallel.c | 6 +-
src/backend/executor/execUtils.c | 1 +
src/backend/executor/functions.c | 7 +-
src/backend/executor/spi.c | 48 ++-
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 346 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/commands/trigger.h | 1 +
src/include/executor/executor.h | 6 +-
src/include/nodes/execnodes.h | 3 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/portal.h | 2 +
24 files changed, 466 insertions(+), 282 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index a46f2db352..58cb62e872 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -330,7 +330,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -967,13 +967,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -996,6 +998,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 0e3547c35b..f7730c8702 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -568,8 +568,11 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 18b07c0200..4a950c03ff 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -329,8 +329,13 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 281c47b2ee..8d1fe5738b 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to have been invalidated after
+ * calling ExecutorStart().
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4873,6 +4905,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b287a2e84c..127d2a3b0a 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -802,7 +802,11 @@ execute_sql_string(const char *sql)
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 22b8b820c3..7083fb2350 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -412,8 +412,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..a1ee5c0acd 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,11 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here,
+ * because there's no CachedPlan.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..f8d0b0ee25 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal has a cached plan and
+ * it's found to be invalidated during the initialization of its plan
+ * trees, the plan must be regenerated.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +647,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 52177759ab..dd139432b9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -5009,6 +5009,19 @@ AfterTriggerBeginQuery(void)
afterTriggers.query_depth++;
}
+/* ----------
+ * AfterTriggerCancelQuery()
+ *
+ * Called from ExecutorEnd() if the query execution was canceled.
+ * ----------
+ */
+void
+AfterTriggerCancelQuery(void)
+{
+ /* Set to a value denoting that no query is active. */
+ afterTriggers.query_depth = -1;
+}
+
/* ----------
* AfterTriggerEndQuery()
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index de7bf7ca67..5755336abd 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,14 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- (void) InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
+
+ /* Mark execution as canceled if plan won't be executed. */
+ estate->es_canceled = !plan_valid;
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -325,6 +338,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
estate = queryDesc->estate;
Assert(estate != NULL);
+ Assert(!estate->es_canceled);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -429,7 +443,7 @@ standard_ExecutorFinish(QueryDesc *queryDesc)
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/* This should be run once and only once per Executor instance */
- Assert(!estate->es_finished);
+ Assert(!estate->es_finished && !estate->es_canceled);
/* Switch into per-query memory context */
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
@@ -488,11 +502,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
Assert(estate != NULL);
/*
- * Check that ExecutorFinish was called, unless in EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
- Assert(estate->es_finished ||
+ Assert(estate->es_finished || estate->es_canceled ||
(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -506,6 +520,14 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
UnregisterSnapshot(estate->es_snapshot);
UnregisterSnapshot(estate->es_crosscheck_snapshot);
+ /*
+ * Cancel trigger execution too if the query execution was canceled.
+ */
+ if (estate->es_canceled &&
+ !(estate->es_top_eflags &
+ (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
+ AfterTriggerCancelQuery();
+
/*
* Must switch out of context before destroying it
*/
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 457ee46faf..13d2820a41 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1437,7 +1437,11 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 16704c0c2f..f0f5740c26 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -151,6 +151,7 @@ CreateExecutorState(void)
estate->es_top_eflags = 0;
estate->es_instrument = 0;
estate->es_finished = false;
+ estate->es_canceled = false;
estate->es_exprcontexts = NIL;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 7e452ed743..606da72535 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -863,7 +863,12 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index f2cca807ef..814ff1390f 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2675,8 +2687,23 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2851,10 +2878,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2898,14 +2924,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 21b9763183..4f923bbcae 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1230,7 +1230,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1735,6 +1740,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2026,9 +2032,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 4ef349df8b..fcf9925ed4 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -118,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, NULL, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -428,19 +343,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -450,15 +367,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -474,6 +389,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -491,8 +408,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
NULL,
@@ -504,30 +421,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -539,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -584,7 +499,82 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ NULL,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -597,19 +587,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1196,7 +1187,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1217,9 +1208,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1236,33 +1228,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1271,26 +1256,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1345,12 +1343,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 430e3ca7dd..d4f7c29301 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -257,6 +257,7 @@ extern void ExecASTruncateTriggers(EState *estate,
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
+extern void AfterTriggerCancelQuery(void);
extern void AfterTriggerEndQuery(EState *estate);
extern void AfterTriggerFireDeferred(void);
extern void AfterTriggerEndXact(bool isCommit);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 72cbf120c5..10c5cda169 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 846eb32a1d..bb5734edb5 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -670,6 +670,9 @@ typedef struct EState
int es_top_eflags; /* eflags passed to ExecutorStart */
int es_instrument; /* OR of InstrumentOption flags */
bool es_finished; /* true when ExecutorFinish is done */
+ bool es_canceled; /* true when execution was canceled
+ * upon encountering that plan was invalided
+ * during ExecInitNode() */
List *es_exprcontexts; /* List of ExprContexts within EState */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v48-0001-Assorted-tightening-in-various-ExecEnd-routines.patch (31.0K, 8-v48-0001-Assorted-tightening-in-various-ExecEnd-routines.patch)
download | inline diff:
From 96983c4519fe018b10b0b8517d205cdebfcb95a2 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 28 Sep 2023 16:56:29 +0900
Subject: [PATCH v48 1/8] Assorted tightening in various ExecEnd()* routines
This includes adding NULLness checks on pointers before cleaning them
in up. Many ExecEnd*() routines already perform this check, but a few
instances remain. These NULLness checks might seem redundant as
things stand since the ExecEnd*() routines operate under the
assumption that their matching ExecInit* routine would have fully
executed, ensuring pointers are set. However, a forthcoming patch will
modify ExecInit* routines to sometimes exit early, potentially leaving
some pointers in an undetermined state, so it will become crucial to
have these NULLness checks in place.
This also adds a guard at the begigging of EvalPlanQualEnd() to return
early if the EPQState does not appear to have been initialized. That
case can happen if the corresponding ExecInit*() routine returned
early without calling EvalPlanQualInit().
While at it, this commit ensures that pointers are consistently set
to NULL after cleanup in all ExecEnd*() routines.
Finally, for enhanced consistency, the format of NULLness checks has
been standardized to "if (pointer != NULL)", replacing the previous
"if (pointer)" style.
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 4 ++
src/backend/executor/nodeAgg.c | 27 +++++++++----
src/backend/executor/nodeAppend.c | 3 ++
src/backend/executor/nodeBitmapAnd.c | 4 +-
src/backend/executor/nodeBitmapHeapscan.c | 47 +++++++++++++++-------
src/backend/executor/nodeBitmapIndexscan.c | 23 +++++------
src/backend/executor/nodeBitmapOr.c | 4 +-
src/backend/executor/nodeForeignscan.c | 17 ++++----
src/backend/executor/nodeGather.c | 1 +
src/backend/executor/nodeGatherMerge.c | 1 +
src/backend/executor/nodeGroup.c | 6 +--
src/backend/executor/nodeHash.c | 6 +--
src/backend/executor/nodeHashjoin.c | 4 +-
src/backend/executor/nodeIncrementalSort.c | 13 +++++-
src/backend/executor/nodeIndexonlyscan.c | 25 ++++++------
src/backend/executor/nodeIndexscan.c | 23 +++++------
src/backend/executor/nodeLimit.c | 1 +
src/backend/executor/nodeLockRows.c | 1 +
src/backend/executor/nodeMaterial.c | 5 ++-
src/backend/executor/nodeMemoize.c | 8 +++-
src/backend/executor/nodeMergeAppend.c | 3 ++
src/backend/executor/nodeMergejoin.c | 2 +
src/backend/executor/nodeModifyTable.c | 11 ++++-
src/backend/executor/nodeNestloop.c | 2 +
src/backend/executor/nodeProjectSet.c | 1 +
src/backend/executor/nodeRecursiveunion.c | 24 +++++++++--
src/backend/executor/nodeResult.c | 1 +
src/backend/executor/nodeSamplescan.c | 7 +++-
src/backend/executor/nodeSeqscan.c | 16 +++-----
src/backend/executor/nodeSetOp.c | 6 ++-
src/backend/executor/nodeSort.c | 5 ++-
src/backend/executor/nodeSubqueryscan.c | 1 +
src/backend/executor/nodeTableFuncscan.c | 4 +-
src/backend/executor/nodeTidrangescan.c | 12 ++++--
src/backend/executor/nodeTidscan.c | 8 +++-
src/backend/executor/nodeUnique.c | 1 +
src/backend/executor/nodeWindowAgg.c | 41 ++++++++++++++-----
37 files changed, 248 insertions(+), 120 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..f7f18d3054 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -3010,6 +3010,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if no EvalPlanQualInit() was done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index f154f28902..af22b1676f 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4314,7 +4313,7 @@ ExecEndAgg(AggState *node)
* worker back into shared memory so that it can be picked up by the main
* process to report in EXPLAIN ANALYZE.
*/
- if (node->shared_info && IsParallelWorker())
+ if (node->shared_info != NULL && IsParallelWorker())
{
AggregateInstrumentation *si;
@@ -4327,10 +4326,16 @@ ExecEndAgg(AggState *node)
/* Make sure we have closed any open tuplesorts */
- if (node->sort_in)
+ if (node->sort_in != NULL)
+ {
tuplesort_end(node->sort_in);
- if (node->sort_out)
+ node->sort_in = NULL;
+ }
+ if (node->sort_out != NULL)
+ {
tuplesort_end(node->sort_out);
+ node->sort_out = NULL;
+ }
hashagg_reset_spill_state(node);
@@ -4346,19 +4351,25 @@ ExecEndAgg(AggState *node)
for (setno = 0; setno < numGroupingSets; setno++)
{
- if (pertrans->sortstates[setno])
+ if (pertrans->sortstates[setno] != NULL)
tuplesort_end(pertrans->sortstates[setno]);
}
}
/* And ensure any agg shutdown callbacks have been called */
for (setno = 0; setno < numGroupingSets; setno++)
+ {
ReScanExprContext(node->aggcontexts[setno]);
- if (node->hashcontext)
+ node->aggcontexts[setno] = NULL;
+ }
+ if (node->hashcontext != NULL)
+ {
ReScanExprContext(node->hashcontext);
+ node->hashcontext = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a2af221e05 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -399,7 +399,10 @@ ExecEndAppend(AppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(appendplans[i]);
+ appendplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..4abb0609a0 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -192,8 +192,8 @@ ExecEndBitmapAnd(BitmapAndState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2db0acfc76..d3f58c22f9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -648,40 +648,59 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
void
ExecEndBitmapHeapScan(BitmapHeapScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * extract information from the node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
+ if (node->tbmiterator != NULL)
+ {
tbm_end_iterate(node->tbmiterator);
- if (node->prefetch_iterator)
+ node->tbmiterator = NULL;
+ }
+ if (node->prefetch_iterator != NULL)
+ {
tbm_end_iterate(node->prefetch_iterator);
- if (node->tbm)
+ node->prefetch_iterator = NULL;
+ }
+ if (node->tbm != NULL)
+ {
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
+ node->tbm = NULL;
+ }
+ if (node->shared_tbmiterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_tbmiterator);
- if (node->shared_prefetch_iterator)
+ node->shared_tbmiterator = NULL;
+ }
+ if (node->shared_prefetch_iterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ node->shared_prefetch_iterator = NULL;
+ }
if (node->vmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->vmbuffer);
+ node->vmbuffer = InvalidBuffer;
+ }
if (node->pvmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->pvmbuffer);
+ node->pvmbuffer = InvalidBuffer;
+ }
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 7cf8532bc9..488f11a3ff 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -175,22 +175,21 @@ ExecReScanBitmapIndexScan(BitmapIndexScanState *node)
void
ExecEndBitmapIndexScan(BitmapIndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->biss_RelationDesc;
- indexScanDesc = node->biss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->biss_ScanDesc != NULL)
+ {
+ index_endscan(node->biss_ScanDesc);
+ node->biss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->biss_RelationDesc != NULL)
+ {
+ index_close(node->biss_RelationDesc, NoLock);
+ node->biss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..ace18593aa 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -210,8 +210,8 @@ ExecEndBitmapOr(BitmapOrState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 73913ebb18..3aba28285a 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -301,17 +301,20 @@ ExecEndForeignScan(ForeignScanState *node)
EState *estate = node->ss.ps.state;
/* Let the FDW shut down */
- if (plan->operation != CMD_SELECT)
+ if (node->fdwroutine != NULL)
{
- if (estate->es_epq_active == NULL)
- node->fdwroutine->EndDirectModify(node);
+ if (plan->operation != CMD_SELECT)
+ {
+ if (estate->es_epq_active == NULL)
+ node->fdwroutine->EndDirectModify(node);
+ }
+ else
+ node->fdwroutine->EndForeignScan(node);
}
- else
- node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index bb2500a469..1a3c8abdad 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -249,6 +249,7 @@ void
ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGather(node);
}
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 7a71a58509..c6fb45fee0 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -289,6 +289,7 @@ void
ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGatherMerge(node);
}
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 8c650f0e46..6dfe5a1d23 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,10 +226,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index e72f0986c2..88ba336882 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,13 +413,11 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* shut down the subplan
*/
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index aea44a9d56..6dc43b9ff2 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -861,7 +861,7 @@ ExecEndHashJoin(HashJoinState *node)
/*
* Free hash table
*/
- if (node->hj_HashTable)
+ if (node->hj_HashTable != NULL)
{
ExecHashTableDestroy(node->hj_HashTable);
node->hj_HashTable = NULL;
@@ -871,7 +871,9 @@ ExecEndHashJoin(HashJoinState *node)
* clean up subtrees
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index cd094a190c..28a0e81cb3 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,8 +1079,16 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ if (node->group_pivot != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->group_pivot);
+ node->group_pivot = NULL;
+ }
+ if (node->transfer_tuple != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ node->transfer_tuple = NULL;
+ }
/*
* Release tuplesort resources.
@@ -1100,6 +1108,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
* Shut down the subplan.
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f1db35665c..1f3843abe9 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -364,15 +364,6 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
void
ExecEndIndexOnlyScan(IndexOnlyScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->ioss_RelationDesc;
- indexScanDesc = node->ioss_ScanDesc;
-
/* Release VM buffer pin, if any. */
if (node->ioss_VMBuffer != InvalidBuffer)
{
@@ -380,13 +371,21 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
+ /* close the scan (no-op if we didn't start it) */
+ if (node->ioss_ScanDesc != NULL)
+ {
+ index_endscan(node->ioss_ScanDesc);
+ node->ioss_ScanDesc = NULL;
+ }
+
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->ioss_RelationDesc != NULL)
+ {
+ index_close(node->ioss_RelationDesc, NoLock);
+ node->ioss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 14b9c00217..32e1714f15 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -785,22 +785,21 @@ ExecIndexAdvanceArrayKeys(IndexArrayKeyInfo *arrayKeys, int numArrayKeys)
void
ExecEndIndexScan(IndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->iss_RelationDesc;
- indexScanDesc = node->iss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->iss_ScanDesc != NULL)
+ {
+ index_endscan(node->iss_ScanDesc);
+ node->iss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->iss_RelationDesc != NULL)
+ {
+ index_close(node->iss_RelationDesc, NoLock);
+ node->iss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 5654158e3e..a97bac9f6d 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,6 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..26fbe95c57 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -387,6 +387,7 @@ ExecEndLockRows(LockRowsState *node)
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 753ea28915..03c514900b 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -243,13 +243,16 @@ ExecEndMaterial(MaterialState *node)
* Release tuplestore resources
*/
if (node->tuplestorestate != NULL)
+ {
tuplestore_end(node->tuplestorestate);
- node->tuplestorestate = NULL;
+ node->tuplestorestate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 94bf479287..ee4749c852 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1043,6 +1043,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable != NULL)
{
int count;
uint64 mem = 0;
@@ -1089,12 +1090,17 @@ ExecEndMemoize(MemoizeState *node)
}
/* Remove the cache context */
- MemoryContextDelete(node->tableContext);
+ if (node->tableContext != NULL)
+ {
+ MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..0a42a04b19 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -333,7 +333,10 @@ ExecEndMergeAppend(MergeAppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(mergeplans[i]);
+ mergeplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index ed3ebe92e5..c84f53e0bd 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1647,7 +1647,9 @@ ExecEndMergeJoin(MergeJoinState *node)
* shut down the subplans
*/
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d21a178ad5..ea043c57c1 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4430,7 +4430,9 @@ ExecEndModifyTable(ModifyTableState *node)
for (j = 0; j < resultRelInfo->ri_NumSlotsInitialized; j++)
{
ExecDropSingleTupleTableSlot(resultRelInfo->ri_Slots[j]);
+ resultRelInfo->ri_Slots[j] = NULL;
ExecDropSingleTupleTableSlot(resultRelInfo->ri_PlanSlots[j]);
+ resultRelInfo->ri_PlanSlots[j] = NULL;
}
}
@@ -4438,12 +4440,16 @@ ExecEndModifyTable(ModifyTableState *node)
* Close all the partitioned tables, leaf partitions, and their indices
* and release the slot used for tuple routing, if set.
*/
- if (node->mt_partition_tuple_routing)
+ if (node->mt_partition_tuple_routing != NULL)
{
ExecCleanupTupleRouting(node, node->mt_partition_tuple_routing);
+ node->mt_partition_tuple_routing = NULL;
- if (node->mt_root_tuple_slot)
+ if (node->mt_root_tuple_slot != NULL)
+ {
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
+ node->mt_root_tuple_slot = NULL;
+ }
}
/*
@@ -4455,6 +4461,7 @@ ExecEndModifyTable(ModifyTableState *node)
* shut down subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index ebd1406843..1211d871ea 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -368,7 +368,9 @@ ExecEndNestLoop(NestLoopState *node)
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index b4bbdc89b1..e9b96416d3 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -324,6 +324,7 @@ ExecEndProjectSet(ProjectSetState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..f6d60bcd6c 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -272,20 +272,36 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table != NULL)
+ {
+ tuplestore_end(node->working_table);
+ node->working_table = NULL;
+ }
+ if (node->intermediate_table != NULL)
+ {
+ tuplestore_end(node->intermediate_table);
+ node->intermediate_table = NULL;
+ }
/* free subsidiary stuff including hashtable */
- if (node->tempContext)
+ if (node->tempContext != NULL)
+ {
MemoryContextDelete(node->tempContext);
- if (node->tableContext)
+ node->tempContext = NULL;
+ }
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index e9f5732f33..f15902e840 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -244,6 +244,7 @@ ExecEndResult(ResultState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 41c1ea37ad..a6813559e6 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -185,14 +185,17 @@ ExecEndSampleScan(SampleScanState *node)
/*
* Tell sampling function that we finished the scan.
*/
- if (node->tsmroutine->EndSampleScan)
+ if (node->tsmroutine != NULL && node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
if (node->ss.ss_currentScanDesc)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 49a5933aff..911266da07 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -183,18 +183,14 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
void
ExecEndSeqScan(SeqScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * get information from node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- if (scanDesc != NULL)
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 98c1b84d43..5c2861d243 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -583,10 +583,14 @@ void
ExecEndSetOp(SetOpState *node)
{
/* free subsidiary stuff including hashtable */
- if (node->tableContext)
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index eea7f2ae15..c8a35b64a8 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -307,13 +307,16 @@ ExecEndSort(SortState *node)
* Release tuplesort resources
*/
if (node->tuplesortstate != NULL)
+ {
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
- node->tuplesortstate = NULL;
+ node->tuplesortstate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 1ee6295660..91d7ae82ce 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -171,6 +171,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
* close down subquery
*/
ExecEndNode(node->subplan);
+ node->subplan = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index a60dcd4943..80ed4b26a8 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -217,8 +217,10 @@ ExecEndTableFuncScan(TableFuncScanState *node)
* Release tuplestore resources
*/
if (node->tupstore != NULL)
+ {
tuplestore_end(node->tupstore);
- node->tupstore = NULL;
+ node->tupstore = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index da622d3f5f..9147e4afa8 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -327,10 +327,14 @@ ExecReScanTidRangeScan(TidRangeScanState *node)
void
ExecEndTidRangeScan(TidRangeScanState *node)
{
- TableScanDesc scan = node->ss.ss_currentScanDesc;
-
- if (scan != NULL)
- table_endscan(scan);
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 15055077d0..74ec6afdcc 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -470,8 +470,14 @@ ExecReScanTidScan(TidScanState *node)
void
ExecEndTidScan(TidScanState *node)
{
- if (node->ss.ss_currentScanDesc)
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 01f951197c..13c556326a 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -169,6 +169,7 @@ void
ExecEndUnique(UniqueState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 77724a6daa..c4c6f009ba 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -1351,11 +1351,14 @@ release_partition(WindowAggState *winstate)
* any aggregate temp data). We don't rely on retail pfree because some
* aggregates might have allocated data we don't have direct pointers to.
*/
- MemoryContextResetAndDeleteChildren(winstate->partcontext);
- MemoryContextResetAndDeleteChildren(winstate->aggcontext);
+ if (winstate->partcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->partcontext);
+ if (winstate->aggcontext != NULL)
+ MemoryContextResetAndDeleteChildren(winstate->aggcontext);
for (i = 0; i < winstate->numaggs; i++)
{
- if (winstate->peragg[i].aggcontext != winstate->aggcontext)
+ if (winstate->peragg[i].aggcontext != NULL &&
+ winstate->peragg[i].aggcontext != winstate->aggcontext)
MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext);
}
@@ -2681,24 +2684,40 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
for (i = 0; i < node->numaggs; i++)
{
- if (node->peragg[i].aggcontext != node->aggcontext)
+ if (node->peragg[i].aggcontext != NULL &&
+ node->peragg[i].aggcontext != node->aggcontext)
MemoryContextDelete(node->peragg[i].aggcontext);
}
- MemoryContextDelete(node->partcontext);
- MemoryContextDelete(node->aggcontext);
+ if (node->partcontext != NULL)
+ {
+ MemoryContextDelete(node->partcontext);
+ node->partcontext = NULL;
+ }
+ if (node->aggcontext != NULL)
+ {
+ MemoryContextDelete(node->aggcontext);
+ node->aggcontext = NULL;
+ }
- pfree(node->perfunc);
- pfree(node->peragg);
+ if (node->perfunc != NULL)
+ {
+ pfree(node->perfunc);
+ node->perfunc = NULL;
+ }
+ if (node->peragg != NULL)
+ {
+ pfree(node->peragg);
+ node->peragg = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* -----------------
--
2.35.3
[application/octet-stream] v48-0002-Prepare-executor-to-support-detecting-CachedPlan.patch (41.7K, 9-v48-0002-Prepare-executor-to-support-detecting-CachedPlan.patch)
download | inline diff:
From a32d7916a4788d958a3dcd7e59a494916d78ddce Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:12:04 +0900
Subject: [PATCH v48 2/8] Prepare executor to support detecting CachedPlan
invalidation
This adds checks at various points during the executor's
initialization of the plan tree to determine whether the originating
CachedPlan has become invalid as a result of taking locks on the
relations referenced in the plan. This includes addding the check
after every call to ExecOpenScanRelation() and to ExecInitNode(),
including the recursive ones to initialize child nodes.
If a given ExecInit*() function detects that the plan has become
invalid, it should return immediately even if the PlanState node
it's building may only be partially valid. That is crucial for
two reasons depending on where the check is:
* The checks following ExecOpenScanRelation() may find the plan
having become invalid because the requested relation was dropped
or had its schema changed concurrently in a manner that risks
unsafe operations in the code that follows. For example, it
might try to dereference a NULL pointer when the check failed
because the relation was dropped.
* For the checks following ExecInitNode(), the returned child
PlanState node might be only partially invalid. The code that
follows may misbehave if it depends on inspecting the child
PlanState. Note that this commit adds the check following all
calls of ExecInitNode() that exist in the code base, even at
sites where there is no code that might misbehave today, because
it might misbehave in the future. It seems like a good idea to
put the guards in place today rather than in the future when the
need arises.
To pass the CachedPlan that the executor will use for these checks,
this adds a new field to QueryDesc and a new parameter to
CreateQueryDesc(). No caller of CreateQueryDesc() is made to pass
an actual CachedPlan though, so there is no functional change.
Reviewed-by: Robert Haas
---
contrib/postgres_fdw/postgres_fdw.c | 10 +++++-
src/backend/commands/copyto.c | 3 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 2 +-
src/backend/commands/extension.c | 1 +
src/backend/commands/matview.c | 2 +-
src/backend/executor/execMain.c | 39 ++++++++++++++++++----
src/backend/executor/execParallel.c | 9 ++++-
src/backend/executor/execProcnode.c | 4 +++
src/backend/executor/functions.c | 1 +
src/backend/executor/nodeAgg.c | 2 ++
src/backend/executor/nodeAppend.c | 10 +++---
src/backend/executor/nodeBitmapAnd.c | 2 ++
src/backend/executor/nodeBitmapHeapscan.c | 4 +++
src/backend/executor/nodeBitmapOr.c | 2 ++
src/backend/executor/nodeCustom.c | 2 ++
src/backend/executor/nodeForeignscan.c | 4 +++
src/backend/executor/nodeGather.c | 2 ++
src/backend/executor/nodeGatherMerge.c | 2 ++
src/backend/executor/nodeGroup.c | 2 ++
src/backend/executor/nodeHash.c | 2 ++
src/backend/executor/nodeHashjoin.c | 4 +++
src/backend/executor/nodeIncrementalSort.c | 2 ++
src/backend/executor/nodeIndexonlyscan.c | 2 ++
src/backend/executor/nodeIndexscan.c | 2 ++
src/backend/executor/nodeLimit.c | 2 ++
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 2 ++
src/backend/executor/nodeMemoize.c | 2 ++
src/backend/executor/nodeMergeAppend.c | 4 ++-
src/backend/executor/nodeMergejoin.c | 4 +++
src/backend/executor/nodeModifyTable.c | 13 ++++++++
src/backend/executor/nodeNestloop.c | 4 +++
src/backend/executor/nodeProjectSet.c | 2 ++
src/backend/executor/nodeRecursiveunion.c | 4 +++
src/backend/executor/nodeResult.c | 2 ++
src/backend/executor/nodeSamplescan.c | 2 ++
src/backend/executor/nodeSeqscan.c | 2 ++
src/backend/executor/nodeSetOp.c | 2 ++
src/backend/executor/nodeSort.c | 2 ++
src/backend/executor/nodeSubqueryscan.c | 2 ++
src/backend/executor/nodeTidrangescan.c | 2 ++
src/backend/executor/nodeTidscan.c | 2 ++
src/backend/executor/nodeUnique.c | 2 ++
src/backend/executor/nodeWindowAgg.c | 2 ++
src/backend/executor/spi.c | 1 +
src/backend/tcop/pquery.c | 5 ++-
src/include/executor/execdesc.h | 4 +++
src/include/executor/executor.h | 10 ++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 ++++++++
51 files changed, 194 insertions(+), 18 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1393716587..0af60463c2 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2126,7 +2126,11 @@ postgresEndForeignModify(EState *estate,
{
PgFdwModifyState *fmstate = (PgFdwModifyState *) resultRelInfo->ri_FdwState;
- /* If fmstate is NULL, we are in EXPLAIN; nothing to do */
+ /*
+ * fmstate could be NULL under two conditions: during an EXPLAIN
+ * operation, or if BeginForeignModify() hasn't been invoked.
+ * In either case, no action is required.
+ */
if (fmstate == NULL)
return;
@@ -2660,7 +2664,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index eaa3172793..0e3547c35b 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 13217807ee..281c47b2ee 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -572,7 +572,7 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
dest = None_Receiver;
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
+ queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..b287a2e84c 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,6 +797,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..22b8b820c3 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,7 +408,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index f7f18d3054..de7bf7ca67 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -263,7 +263,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ (void) InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
}
@@ -829,9 +829,13 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise. The latter case may occur if the CachedPlan that provides
+ * the plan tree (queryDesc->cplan) got invalidated during the initialization.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,11 +843,14 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
- TupleDesc tupType;
+ PlanState *planstate = NULL;
+ TupleDesc tupType = NULL;
ListCell *l;
int i;
+ Assert(queryDesc->planstate == NULL);
+ Assert(queryDesc->tupDesc == NULL);
+
/*
* Do permissions checks
*/
@@ -855,6 +862,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +894,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (unlikely(relation == NULL))
+ return false;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -956,6 +966,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
i++;
}
@@ -966,6 +978,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1009,6 +1023,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
+
+ return true;
}
/*
@@ -2858,7 +2874,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2947,6 +2964,13 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
subplanstate = ExecInitNode(subplan, rcestate, 0);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
+
+ /*
+ * All the necessary locks must already have been taken when
+ * initializing the parent's copy of subplanstate, so the CachedPlan,
+ * if any, should not have become invalid during ExecInitNode().
+ */
+ Assert(ExecPlanStillValid(rcestate));
}
/*
@@ -2988,6 +3012,9 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /* See the comment above. */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..457ee46faf 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Set up a QueryDesc for the query. While the leader might've sourced
+ * the plan tree from a CachedPlan, we don't have one here. This isn't
+ * an issue since the leader ensured the required locks, making our
+ * plan tree valid. Even as we get our own lock copies in
+ * ExecGetRangeTableRelation(), they're all already held by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index b4b5c562c0..febaa194c4 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -136,6 +136,10 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * Callers should check upon returning that ExecPlanStillValid(estate)
+ * returns true before continuing further with its processing, because the
+ * returned PlanState might be only partially valid otherwise.
* ------------------------------------------------------------------------
*/
PlanState *
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..7e452ed743 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index af22b1676f..597d68139e 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return aggstate;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index a2af221e05..53ca9dc85d 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -185,8 +185,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ appendplanstates = (PlanState **) palloc0(nplans *
+ sizeof(PlanState *));
+ appendstate->appendplans = appendplanstates;
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +223,11 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
- appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4abb0609a0..7556be713c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmapandstate;
i++;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d3f58c22f9..f1f8e16b17 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -770,11 +770,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index ace18593aa..7d2bf45d9c 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmaporstate;
i++;
}
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index 28b5bb9353..a0befbd0c6 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return css;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 3aba28285a..336acff719 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 1a3c8abdad..c524022c04 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gatherstate;
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index c6fb45fee0..676faabef5 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 6dfe5a1d23..efa1c44ab4 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return grpstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 88ba336882..1a4bd5504e 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 6dc43b9ff2..c0919074b0 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 28a0e81cb3..621ffafe02 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return incrsortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 1f3843abe9..c555c14888 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -495,6 +495,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 32e1714f15..a3bd1f7fb0 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -908,6 +908,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a97bac9f6d..ab133f1580 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return limitstate;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 26fbe95c57..e1ef768571 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 03c514900b..c38eef099d 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index ee4749c852..a6bf66029c 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 0a42a04b19..52c3edf278 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -120,7 +120,7 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +151,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index c84f53e0bd..887f519b10 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ea043c57c1..95d909c1d0 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3985,6 +3985,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ /*
+ * ExecInitResultRelation() may have returned without initializing
+ * rootResultRelInfo if the plan got invalidated, so check.
+ */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4013,6 +4020,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ /* See the comment above. */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/*
* For child result relations, store the root result relation
* pointer. We do so for the convenience of places that want to
@@ -4039,6 +4050,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 1211d871ea..8d67d17e10 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index e9b96416d3..706cc23a21 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return state;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index f6d60bcd6c..27dc318acb 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index f15902e840..6820d3bfd5 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return resstate;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index a6813559e6..02051fea51 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 911266da07..9e3ef94388 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 5c2861d243..475af4df24 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c8a35b64a8..9de717aa7c 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return sortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 91d7ae82ce..d9c10d1f6f 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 9147e4afa8..a7482aee50 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -378,6 +378,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidrangestate;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 74ec6afdcc..657411ef19 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -523,6 +523,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidstate;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 13c556326a..ee30688417 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index c4c6f009ba..1246d7919a 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2461,6 +2461,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..f2cca807ef 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2668,6 +2668,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ NULL,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..4ef349df8b 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -65,6 +65,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +78,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -145,7 +147,7 @@ ProcessQuery(PlannedStmt *plan,
/*
* Create the QueryDesc object
*/
- queryDesc = CreateQueryDesc(plan, sourceText,
+ queryDesc = CreateQueryDesc(plan, NULL, sourceText,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
@@ -493,6 +495,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ NULL,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index aeebe0e0ff..72cbf120c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,15 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the CachedPlan in es_cachedplan still valid?
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..846eb32a1d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one, or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..0a9e041d51 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Invoked by the executor for each relation lock acquired during the
+ * initialization of the plan tree within the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-11-20 04:29 Amit Langote <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 3 replies; 31+ messages in thread
From: Amit Langote @ 2023-11-20 04:29 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Thu, Sep 28, 2023 at 5:26 PM Amit Langote <[email protected]> wrote:
> On Tue, Sep 26, 2023 at 10:06 PM Amit Langote <[email protected]> wrote:
> > After sleeping on this, I think we do need the checks after all the
> > ExecInitNode() calls too, because we have many instances of the code
> > like the following one:
> >
> > outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
> > tupDesc = ExecGetResultType(outerPlanState(gatherstate));
> > <some code that dereferences outDesc>
> >
> > If outerNode is a SeqScan and ExecInitSeqScan() returned early because
> > ExecOpenScanRelation() detected that plan was invalidated, then
> > tupDesc would be NULL in this case, causing the code to crash.
> >
> > Now one might say that perhaps we should only add the
> > is-CachedPlan-valid test in the instances where there is an actual
> > risk of such misbehavior, but that could lead to confusion, now or
> > later. It seems better to add them after every ExecInitNode() call
> > while we're inventing the notion, because doing so relieves the
> > authors of future enhancements of the ExecInit*() routines from
> > worrying about any of this.
> >
> > Attached 0003 should show how that turned out.
> >
> > Updated 0002 as mentioned in the previous reply -- setting pointers to
> > NULL after freeing them more consistently across various ExecEnd*()
> > routines and using the `if (pointer != NULL)` style over the `if
> > (pointer)` more consistently.
> >
> > Updated 0001's commit message to remove the mention of its relation to
> > any future commits. I intend to push it tomorrow.
>
> Pushed that one. Here are the rebased patches.
>
> 0001 seems ready to me, but I'll wait a couple more days for others to
> weigh in. Just to highlight a kind of change that others may have
> differing opinions on, consider this hunk from the patch:
>
> - MemoryContextDelete(node->aggcontext);
> + if (node->aggcontext != NULL)
> + {
> + MemoryContextDelete(node->aggcontext);
> + node->aggcontext = NULL;
> + }
> ...
> + ExecEndNode(outerPlanState(node));
> + outerPlanState(node) = NULL;
>
> So the patch wants to enhance the consistency of setting the pointer
> to NULL after freeing part. Robert mentioned his preference for doing
> it in the patch, which I agree with.
Rebased.
I haven't been able to reproduce and debug a crash reported by cfbot
that I see every now and then:
https://cirrus-ci.com/task/5673432591892480?logs=cores#L0
[22:46:12.328] Program terminated with signal SIGSEGV, Segmentation fault.
[22:46:12.328] Address not mapped to object.
[22:46:12.838] #0 afterTriggerInvokeEvents
(events=events@entry=0x836db0460, firing_id=1,
estate=estate@entry=0x842eec100, delete_ok=<optimized out>) at
../src/backend/commands/trigger.c:4656
[22:46:12.838] #1 0x00000000006c67a8 in AfterTriggerEndQuery
(estate=estate@entry=0x842eec100) at
../src/backend/commands/trigger.c:5085
[22:46:12.838] #2 0x000000000065bfba in CopyFrom (cstate=0x836df9038)
at ../src/backend/commands/copyfrom.c:1293
...
While a patch in this series does change
src/backend/commands/trigger.c, I'm not yet sure about its relation
with the backtrace shown there.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v49-0006-Add-field-to-store-parent-relids-to-Append-Merge.patch (26.1K, 2-v49-0006-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From bbde32f56bfbaba07d699dec4838f21d43e40384 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:02 +0900
Subject: [PATCH v49 6/8] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo. A future commit will create a need for them to
be available at all times for the purpose of locking those
parent/ancestor tables when executing a cached plan, so add a
field called allpartrelids to Append/MergeAppend to store those
RT indexes. This also adds a function called
ExecLockAppendNonLeafTables() to lock those tables.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 2 +-
src/backend/executor/execUtils.c | 33 ++++++
src/backend/executor/nodeAppend.c | 14 +++
src/backend/executor/nodeMergeAppend.c | 14 +++
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/executor/executor.h | 1 +
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
13 files changed, 266 insertions(+), 124 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index ffc62e379a..2804ec70f1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1475,7 +1475,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafPartitions().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 117773706a..2b7a08c9ba 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -827,6 +827,39 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafPartitions
+ * Lock non-leaf partitions whose child partitions are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i = -1;
+
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ /*
+ * Don't lock the root parent mentioned in the query, because it
+ * should already have been locked before entering the executor.
+ */
+ if (!rte->inFromCl)
+ LockRelationOid(rte->relid, rte->rellockmode);
+ else
+ Assert(CheckRelLockedByMe(rte->relid, rte->rellockmode, true));
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 53ca9dc85d..4759511f87 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,20 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 52c3edf278..158210aac1 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,20 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ ExecLockAppendNonLeafPartitions(estate, node->allpartrelids);
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..d1f4f606bf 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1229,6 +1230,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1370,15 +1372,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1399,7 +1409,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1445,6 +1456,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1534,15 +1546,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1554,7 +1574,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index a8cea5efe1..63a11e511e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7850,8 +7850,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index fc3709510d..04910c301f 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 3f31ecc956..42461a5b2c 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -216,33 +215,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -251,50 +249,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -360,63 +317,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 4f183ec6cd..350a625107 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -601,6 +601,7 @@ exec_rt_fetch(Index rti, EState *estate)
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
+extern void ExecLockAppendNonLeafPartitions(EState *estate, List *allpartrelids);
extern int executor_errposition(EState *estate, int location);
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index d40af8e59f..ffcc8490d7 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v49-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch (25.2K, 3-v49-0007-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From a8bc206d5367decfb04f95283a67645b85ba5c81 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:15 +0900
Subject: [PATCH v49 7/8] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
Previous commits have made all the necessary adjustment to make the
executor lock child tables, to detect invalidation of the CachedPlan
resulting from that, and to retry the execution with a new CachePlan.
So, this commit simply removes the code in plancache.c that does the
"for execution" locking, aka AcquireExecutorLocks().
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/spi.c | 2 +-
src/backend/tcop/pquery.c | 6 +-
src/backend/utils/cache/plancache.c | 154 +++++++----------
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 158 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
7 files changed, 343 insertions(+), 108 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 60e2632cd2..99f4ed92fb 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2680,7 +2680,7 @@ replan:
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
- NULL,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index fcf9925ed4..8d0772ae29 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -412,7 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
- NULL,
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -443,6 +443,7 @@ PortalStart(Portal portal, ParamListInfo params,
*/
if (!ExecutorStart(queryDesc, myeflags))
{
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
PopActiveSnapshot();
@@ -542,7 +543,7 @@ PortalStart(Portal portal, ParamListInfo params,
* PortalRunMulti() before calling ExecutorRun().
*/
queryDesc = CreateQueryDesc(plan,
- NULL,
+ portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -566,6 +567,7 @@ PortalStart(Portal portal, ParamListInfo params,
if (!ExecutorStart(queryDesc, myeflags))
{
PopActiveSnapshot();
+ Assert(queryDesc->cplan);
ExecutorEnd(queryDesc);
FreeQueryDesc(queryDesc);
plan_valid = false;
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 8f95520f2f..fad1457192 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -104,13 +104,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -817,8 +817,13 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan includes child relations introduced by the planner, they
+ * wouldn't be locked yet. This is because AcquirePlannerLocks() only locks
+ * relations present in the original query's range table (before planner
+ * entry). Hence, the plan might become stale if child relations are modified
+ * concurrently. During the plan initialization, the executor must ensure the
+ * plan (CachedPlan) remains valid after locking each child table. If found
+ * invalid, the caller should be prompted to recreate the plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -832,60 +837,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1155,8 +1156,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * Typically, the plan returned by this function is valid. However, a caveat
+ * arises with inheritance/partition child tables. These aren't locked by
+ * this function, as we only lock tables directly mentioned in the original
+ * query here. The task of locking these child tables falls to the executor
+ * during plan tree setup. If acquiring these locks invalidates the plan, the
+ * executor should inform the caller to regenerate the plan by invoking this
+ * function again. The reason for this deferred child table locking mechanism
+ * is efficiency: not all might need to be locked. Some could be pruned during
+ * executor initialization, especially if their corresponding plan nodes
+ * facilitate partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1191,7 +1200,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1389,8 +1401,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1766,58 +1778,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..122d81f2ee
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,158 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+----------------------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = $1)
+(7 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------------
+LockRows
+ -> Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(5 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..2d0607b176
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo11 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1 FOR UPDATE;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v49-0005-Assert-that-relations-needing-their-permissions-.patch (5.1K, 4-v49-0005-Assert-that-relations-needing-their-permissions-.patch)
download | inline diff:
From 703d7f12172a74c2ef26fa7723105846b4365b4c Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Mon, 25 Sep 2023 11:52:02 +0900
Subject: [PATCH v49 5/8] Assert that relations needing their permissions
checked are locked
---
src/backend/executor/execMain.c | 11 +++++++
src/backend/storage/lmgr/lmgr.c | 45 +++++++++++++++++++++++++++++
src/backend/utils/cache/lsyscache.c | 21 ++++++++++++++
src/include/storage/lmgr.h | 1 +
src/include/utils/lsyscache.h | 1 +
5 files changed, 79 insertions(+)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5755336abd..ffc62e379a 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -626,6 +626,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
(rte->rtekind == RTE_SUBQUERY &&
rte->relkind == RELKIND_VIEW));
+ /*
+ * Relations whose permissions need to be checked must already
+ * have been locked by the parser or by GetCachedPlan() if a
+ * cached plan is being executed.
+ *
+ * XXX Maybe we should we skip calling ExecCheckPermissions from
+ * InitPlan in a parallel worker.
+ */
+ Assert(IsParallelWorker() ||
+ CheckRelLockedByMe(rte->relid, AccessShareLock, true));
+
(void) getRTEPermissionInfo(rteperminfos, rte);
/* Many-to-one mapping not allowed */
Assert(!bms_is_member(rte->perminfoindex, indexset));
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index b447ddf11b..3c05d8d87d 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 39f0e346b0..426614050d 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index c22cabdf42..177e5d7f12 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
--
2.35.3
[application/octet-stream] v49-0004-Teach-the-executor-to-lock-child-tables-in-some-.patch (11.1K, 5-v49-0004-Teach-the-executor-to-lock-child-tables-in-some-.patch)
download | inline diff:
From dce6657e911afecef82cddda90e0d236c273fe18 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:17:15 +0900
Subject: [PATCH v49 4/8] Teach the executor to lock child tables in some cases
An upcoming commit will move the locking of child tables referenced
in a cached plan tree from GetCachedPlan() to the executor
initialization of the plan tree in ExecutorStart(). This commit
teaches ExecGetRangeTableRelation() to lock child tables if
EState.es_cachedplan points to a CachedPlan.
The executor must now deal with the cases where an unlocked child
table might have been concurrently dropped, so this modifies
ExecGetRangeTableRelation() to use try_table_open(). All of its
callers (and those of ExecOpenScanRelation() that calls it) must
now account for the child table disappearing, which means to abort
initializing the table's Scan node in the middle.
ExecGetRangeTableRelation() now examines inFromCl field of an RTE
to determine that a given range table relation is a child table, so
this commit also makes the planner set inFromCl to false in the
child tables' RTEs that it manufactures.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/executor/README | 36 +++++++++++++++++++++++-
src/backend/executor/execPartition.c | 2 ++
src/backend/executor/execUtils.c | 41 +++++++++++++++++++++-------
src/backend/optimizer/util/inherit.c | 7 +++++
src/backend/parser/analyze.c | 7 ++---
src/include/nodes/parsenodes.h | 8 ++++--
6 files changed, 84 insertions(+), 17 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 642d63be61..6cd840d3a7 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,34 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Typically, when the executor initializes a plan tree for execution, it doesn't
+lock non-index relations if the plan tree is freshly generated and not derived
+from a CachedPlan. This is because such locks have already been established
+during the query's parsing, rewriting, and planning phases. However, with a
+cached plan tree, there can be relations that remain unlocked. The function
+GetCachedPlan() locks relations existing in the query's range table pre-planning
+but doesn't account for those added during the planning phase. Consequently,
+inheritance child tables, introduced to the query's range table during planning,
+won't be locked when the cached plan reaches the executor.
+
+The decision to defer locking child tables with GetCachedPlan() arises from the
+fact that not all might be accessed during plan execution. For instance, if
+child tables are partitions, some might be omitted due to pruning at
+execution-initialization-time. Thus, the responsibility of locking these child
+tables is pushed to execution-initialization-time, taking place in ExecInitNode()
+for plan nodes encompassing these tables.
+
+This approach opens a window where a cached plan tree with child tables could
+become outdated if another backend modifies these tables before ExecInitNode()
+locks them. Given this, the executor has the added duty to confirm the plan
+tree's validity whenever it locks a child table post execution-initialization-
+pruning. This validation is done by checking the CachedPlan.is_valid attribute
+of the CachedPlan provided. If the plan tree is outdated (is_valid=false), the
+executor halts any further initialization and alerts the caller that they should
+retry execution with another freshly created plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,7 +344,13 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index f6c34328b8..532734d758 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -1927,6 +1927,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (unlikely(partrel == NULL))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f0f5740c26..117773706a 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -697,6 +697,8 @@ ExecRelationIsTargetRelation(EState *estate, Index scanrelid)
*
* Open the heap relation to be scanned by a base-level scan plan node.
* This should be called during the node's ExecInit routine.
+ *
+ * NULL is returned if the relation is found to have been dropped.
* ----------------------------------------------------------------
*/
Relation
@@ -706,6 +708,8 @@ ExecOpenScanRelation(EState *estate, Index scanrelid, int eflags)
/* Open the relation. */
rel = ExecGetRangeTableRelation(estate, scanrelid);
+ if (unlikely(rel == NULL))
+ return NULL;
/*
* Complain if we're attempting a scan of an unscannable relation, except
@@ -763,6 +767,9 @@ ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos)
* Open the Relation for a range table entry, if not already done
*
* The Relations will be closed again in ExecEndPlan().
+ *
+ * Returned value may be NULL if the relation is a child relation that is not
+ * already locked.
*/
Relation
ExecGetRangeTableRelation(EState *estate, Index rti)
@@ -779,7 +786,28 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ *
+ * Note that we use try_table_open() here, because without a lock
+ * held on the relation, it may have disappeared from under us.
+ */
+ rel = try_table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -792,15 +820,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -823,6 +842,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (unlikely(resultRelationDesc == NULL))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index f9d3ff1e7a..1b9d79e341 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -493,6 +493,13 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+
+ /*
+ * Flag child tables as indirectly referenced in the query. This helps
+ * the executor's ExecGetRangeTableRelation() recognize them as
+ * inheritance children.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/* No permission checking for child RTEs. */
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 7a1dfb6364..cf269f8c53 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3305,10 +3305,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e494309da8..642c7bdfea 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -987,11 +987,15 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. The executor uses
+ * this to ascertain if an RTE_RELATION entry is for a table explicitly
+ * named in the query or a child table added by the planner. This
+ * distinction is vital when child tables in a plan must be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v49-0008-Track-opened-range-table-relations-in-a-List-in-.patch (2.5K, 6-v49-0008-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From bae4dc2096b673c1ff8ff7da2f4e068ae39d859e Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:54:19 +0900
Subject: [PATCH v49 8/8] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 3 +++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 2804ec70f1..d559c1de61 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1649,12 +1649,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 2b7a08c9ba..1dfef44495 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -822,6 +822,9 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ if (rel != NULL)
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cad2329ac9..c7bd235737 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -618,6 +618,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v49-0002-Prepare-executor-to-support-detecting-CachedPlan.patch (41.7K, 7-v49-0002-Prepare-executor-to-support-detecting-CachedPlan.patch)
download | inline diff:
From 1c134f99b792c8421d9d5f9590c2f761c6dd31c7 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Fri, 22 Sep 2023 18:12:04 +0900
Subject: [PATCH v49 2/8] Prepare executor to support detecting CachedPlan
invalidation
This adds checks at various points during the executor's
initialization of the plan tree to determine whether the originating
CachedPlan has become invalid as a result of taking locks on the
relations referenced in the plan. This includes addding the check
after every call to ExecOpenScanRelation() and to ExecInitNode(),
including the recursive ones to initialize child nodes.
If a given ExecInit*() function detects that the plan has become
invalid, it should return immediately even if the PlanState node
it's building may only be partially valid. That is crucial for
two reasons depending on where the check is:
* The checks following ExecOpenScanRelation() may find the plan
having become invalid because the requested relation was dropped
or had its schema changed concurrently in a manner that risks
unsafe operations in the code that follows. For example, it
might try to dereference a NULL pointer when the check failed
because the relation was dropped.
* For the checks following ExecInitNode(), the returned child
PlanState node might be only partially invalid. The code that
follows may misbehave if it depends on inspecting the child
PlanState. Note that this commit adds the check following all
calls of ExecInitNode() that exist in the code base, even at
sites where there is no code that might misbehave today, because
it might misbehave in the future. It seems like a good idea to
put the guards in place today rather than in the future when the
need arises.
To pass the CachedPlan that the executor will use for these checks,
this adds a new field to QueryDesc and a new parameter to
CreateQueryDesc(). No caller of CreateQueryDesc() is made to pass
an actual CachedPlan though, so there is no functional change.
Reviewed-by: Robert Haas
---
contrib/postgres_fdw/postgres_fdw.c | 10 +++++-
src/backend/commands/copyto.c | 3 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 2 +-
src/backend/commands/extension.c | 1 +
src/backend/commands/matview.c | 2 +-
src/backend/executor/execMain.c | 39 ++++++++++++++++++----
src/backend/executor/execParallel.c | 9 ++++-
src/backend/executor/execProcnode.c | 4 +++
src/backend/executor/functions.c | 1 +
src/backend/executor/nodeAgg.c | 2 ++
src/backend/executor/nodeAppend.c | 10 +++---
src/backend/executor/nodeBitmapAnd.c | 2 ++
src/backend/executor/nodeBitmapHeapscan.c | 4 +++
src/backend/executor/nodeBitmapOr.c | 2 ++
src/backend/executor/nodeCustom.c | 2 ++
src/backend/executor/nodeForeignscan.c | 4 +++
src/backend/executor/nodeGather.c | 2 ++
src/backend/executor/nodeGatherMerge.c | 2 ++
src/backend/executor/nodeGroup.c | 2 ++
src/backend/executor/nodeHash.c | 2 ++
src/backend/executor/nodeHashjoin.c | 4 +++
src/backend/executor/nodeIncrementalSort.c | 2 ++
src/backend/executor/nodeIndexonlyscan.c | 2 ++
src/backend/executor/nodeIndexscan.c | 2 ++
src/backend/executor/nodeLimit.c | 2 ++
src/backend/executor/nodeLockRows.c | 2 ++
src/backend/executor/nodeMaterial.c | 2 ++
src/backend/executor/nodeMemoize.c | 2 ++
src/backend/executor/nodeMergeAppend.c | 4 ++-
src/backend/executor/nodeMergejoin.c | 4 +++
src/backend/executor/nodeModifyTable.c | 13 ++++++++
src/backend/executor/nodeNestloop.c | 4 +++
src/backend/executor/nodeProjectSet.c | 2 ++
src/backend/executor/nodeRecursiveunion.c | 4 +++
src/backend/executor/nodeResult.c | 2 ++
src/backend/executor/nodeSamplescan.c | 2 ++
src/backend/executor/nodeSeqscan.c | 2 ++
src/backend/executor/nodeSetOp.c | 2 ++
src/backend/executor/nodeSort.c | 2 ++
src/backend/executor/nodeSubqueryscan.c | 2 ++
src/backend/executor/nodeTidrangescan.c | 2 ++
src/backend/executor/nodeTidscan.c | 2 ++
src/backend/executor/nodeUnique.c | 2 ++
src/backend/executor/nodeWindowAgg.c | 2 ++
src/backend/executor/spi.c | 1 +
src/backend/tcop/pquery.c | 5 ++-
src/include/executor/execdesc.h | 4 +++
src/include/executor/executor.h | 10 ++++++
src/include/nodes/execnodes.h | 2 ++
src/include/utils/plancache.h | 14 ++++++++
51 files changed, 194 insertions(+), 18 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 6de2bec3b7..491200a1fd 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2126,7 +2126,11 @@ postgresEndForeignModify(EState *estate,
{
PgFdwModifyState *fmstate = (PgFdwModifyState *) resultRelInfo->ri_FdwState;
- /* If fmstate is NULL, we are in EXPLAIN; nothing to do */
+ /*
+ * fmstate could be NULL under two conditions: during an EXPLAIN
+ * operation, or if BeginForeignModify() hasn't been invoked.
+ * In either case, no action is required.
+ */
if (fmstate == NULL)
return;
@@ -2660,7 +2664,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index c66a047c4a..0929ad929a 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..18b07c0200 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f1d71bc54e..c698e54fec 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -572,7 +572,7 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
dest = None_Receiver;
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
+ queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 535072d181..b287a2e84c 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,6 +797,7 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..22b8b820c3 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,7 +408,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index f7f18d3054..de7bf7ca67 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(PlanState *planstate, EState *estate);
@@ -263,7 +263,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ (void) InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
}
@@ -829,9 +829,13 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise. The latter case may occur if the CachedPlan that provides
+ * the plan tree (queryDesc->cplan) got invalidated during the initialization.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,11 +843,14 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
- TupleDesc tupType;
+ PlanState *planstate = NULL;
+ TupleDesc tupType = NULL;
ListCell *l;
int i;
+ Assert(queryDesc->planstate == NULL);
+ Assert(queryDesc->tupDesc == NULL);
+
/*
* Do permissions checks
*/
@@ -855,6 +862,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
estate->es_plannedstmt = plannedstmt;
+ estate->es_cachedplan = queryDesc->cplan;
/*
* Next, build the ExecRowMark array from the PlanRowMark(s), if any.
@@ -886,6 +894,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (unlikely(relation == NULL))
+ return false;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -956,6 +966,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
i++;
}
@@ -966,6 +978,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return false;
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1009,6 +1023,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
queryDesc->tupDesc = tupType;
queryDesc->planstate = planstate;
+
+ return true;
}
/*
@@ -2858,7 +2874,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2947,6 +2964,13 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
subplanstate = ExecInitNode(subplan, rcestate, 0);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
+
+ /*
+ * All the necessary locks must already have been taken when
+ * initializing the parent's copy of subplanstate, so the CachedPlan,
+ * if any, should not have become invalid during ExecInitNode().
+ */
+ Assert(ExecPlanStillValid(rcestate));
}
/*
@@ -2988,6 +3012,9 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /* See the comment above. */
+ Assert(ExecPlanStillValid(rcestate));
+
MemoryContextSwitchTo(oldcontext);
}
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..457ee46faf 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Set up a QueryDesc for the query. While the leader might've sourced
+ * the plan tree from a CachedPlan, we don't have one here. This isn't
+ * an issue since the leader ensured the required locks, making our
+ * plan tree valid. Even as we get our own lock copies in
+ * ExecGetRangeTableRelation(), they're all already held by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index b4b5c562c0..febaa194c4 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -136,6 +136,10 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * Callers should check upon returning that ExecPlanStillValid(estate)
+ * returns true before continuing further with its processing, because the
+ * returned PlanState might be only partially valid otherwise.
* ------------------------------------------------------------------------
*/
PlanState *
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index bace25234c..66636b05a5 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index af22b1676f..597d68139e 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return aggstate;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index a2af221e05..53ca9dc85d 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -185,8 +185,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->ps.resultopsset = true;
appendstate->ps.resultopsfixed = false;
- appendplanstates = (PlanState **) palloc(nplans *
- sizeof(PlanState *));
+ appendplanstates = (PlanState **) palloc0(nplans *
+ sizeof(PlanState *));
+ appendstate->appendplans = appendplanstates;
+ appendstate->as_nplans = nplans;
/*
* call ExecInitNode on each of the valid plans to be executed and save
@@ -221,11 +223,11 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return appendstate;
}
appendstate->as_first_partial_plan = firstvalid;
- appendstate->appendplans = appendplanstates;
- appendstate->as_nplans = nplans;
/* Initialize async state */
appendstate->as_asyncplans = asyncplans;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4abb0609a0..7556be713c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmapandstate;
i++;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d3f58c22f9..f1f8e16b17 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -770,11 +770,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index ace18593aa..7d2bf45d9c 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
{
initNode = (Plan *) lfirst(l);
bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return bitmaporstate;
i++;
}
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index 28b5bb9353..a0befbd0c6 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return css;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 3aba28285a..336acff719 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 1a3c8abdad..c524022c04 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gatherstate;
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index c6fb45fee0..676faabef5 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return gm_state;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 6dfe5a1d23..efa1c44ab4 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return grpstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 88ba336882..1a4bd5504e 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hashstate;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 03dd931527..e3b3c2305f 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return hjstate;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 28a0e81cb3..621ffafe02 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return incrsortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 1f3843abe9..c555c14888 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -495,6 +495,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 32e1714f15..a3bd1f7fb0 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -908,6 +908,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return indexstate;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a97bac9f6d..ab133f1580 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return limitstate;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 26fbe95c57..e1ef768571 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return lrstate;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 03c514900b..c38eef099d 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return matstate;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 9b13e2e552..dfd695ad07 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -957,6 +957,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mstate;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 0a42a04b19..52c3edf278 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -120,7 +120,7 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ms_prune_state = NULL;
}
- mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *));
+ mergeplanstates = (PlanState **) palloc0(nplans * sizeof(PlanState *));
mergestate->mergeplans = mergeplanstates;
mergestate->ms_nplans = nplans;
@@ -151,6 +151,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 4b181621f9..e5d4f8e21d 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mergestate;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index bc82fb033a..e75500a1f5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4002,6 +4002,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ /*
+ * ExecInitResultRelation() may have returned without initializing
+ * rootResultRelInfo if the plan got invalidated, so check.
+ */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4030,6 +4037,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ /* See the comment above. */
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
+
/*
* For child result relations, store the root result relation
* pointer. We do so for the convenience of places that want to
@@ -4056,6 +4067,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return mtstate;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 1211d871ea..8d67d17e10 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return nlstate;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index 514669558c..2074ba6683 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -255,6 +255,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return state;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index 08637a64db..f07205f958 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return rustate;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index f15902e840..6820d3bfd5 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return resstate;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index a6813559e6..02051fea51 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 911266da07..9e3ef94388 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return scanstate;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 92bd2da8e0..46e294ba52 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return setopstate;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c8a35b64a8..9de717aa7c 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return sortstate;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 91d7ae82ce..d9c10d1f6f 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return subquerystate;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index d6b4ed2e42..831f514c4d 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -378,6 +378,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidrangestate;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 74ec6afdcc..657411ef19 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -523,6 +523,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return tidstate;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 13c556326a..ee30688417 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return uniquestate;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 0253c47448..809cedf187 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2461,6 +2461,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (unlikely(!ExecPlanStillValid(estate)))
+ return winstate;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 0e46c59d25..892b2853ed 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2668,6 +2668,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ NULL,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..4ef349df8b 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -65,6 +65,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -77,6 +78,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -145,7 +147,7 @@ ProcessQuery(PlannedStmt *plan,
/*
* Create the QueryDesc object
*/
- queryDesc = CreateQueryDesc(plan, sourceText,
+ queryDesc = CreateQueryDesc(plan, NULL, sourceText,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
@@ -493,6 +495,7 @@ PortalStart(Portal portal, ParamListInfo params,
* the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ NULL,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index e1eefb400b..3b33b38196 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -256,6 +257,15 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the CachedPlan in es_cachedplan still valid?
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 5d7f17dee0..08670bc5ed 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -622,6 +622,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one, or NULL if not */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 110e649fce..5a54f8a917 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -223,6 +223,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Invoked by the executor for each relation lock acquired during the
+ * initialization of the plan tree within the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
--
2.35.3
[application/octet-stream] v49-0001-Assorted-tightening-in-various-ExecEnd-routines.patch (30.9K, 8-v49-0001-Assorted-tightening-in-various-ExecEnd-routines.patch)
download | inline diff:
From f7902c69704598e9dbbc1ee72d42bc9ca6c98a21 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 28 Sep 2023 16:56:29 +0900
Subject: [PATCH v49 1/8] Assorted tightening in various ExecEnd()* routines
This includes adding NULLness checks on pointers before cleaning them
in up. Many ExecEnd*() routines already perform this check, but a few
instances remain. These NULLness checks might seem redundant as
things stand since the ExecEnd*() routines operate under the
assumption that their matching ExecInit* routine would have fully
executed, ensuring pointers are set. However, a forthcoming patch will
modify ExecInit* routines to sometimes exit early, potentially leaving
some pointers in an undetermined state, so it will become crucial to
have these NULLness checks in place.
This also adds a guard at the begigging of EvalPlanQualEnd() to return
early if the EPQState does not appear to have been initialized. That
case can happen if the corresponding ExecInit*() routine returned
early without calling EvalPlanQualInit().
While at it, this commit ensures that pointers are consistently set
to NULL after cleanup in all ExecEnd*() routines.
Finally, for enhanced consistency, the format of NULLness checks has
been standardized to "if (pointer != NULL)", replacing the previous
"if (pointer)" style.
Reviewed-by: Robert Haas
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 4 ++
src/backend/executor/nodeAgg.c | 27 +++++++++----
src/backend/executor/nodeAppend.c | 3 ++
src/backend/executor/nodeBitmapAnd.c | 4 +-
src/backend/executor/nodeBitmapHeapscan.c | 47 +++++++++++++++-------
src/backend/executor/nodeBitmapIndexscan.c | 23 +++++------
src/backend/executor/nodeBitmapOr.c | 4 +-
src/backend/executor/nodeForeignscan.c | 17 ++++----
src/backend/executor/nodeGather.c | 1 +
src/backend/executor/nodeGatherMerge.c | 1 +
src/backend/executor/nodeGroup.c | 6 +--
src/backend/executor/nodeHash.c | 6 +--
src/backend/executor/nodeHashjoin.c | 4 +-
src/backend/executor/nodeIncrementalSort.c | 13 +++++-
src/backend/executor/nodeIndexonlyscan.c | 25 ++++++------
src/backend/executor/nodeIndexscan.c | 23 +++++------
src/backend/executor/nodeLimit.c | 1 +
src/backend/executor/nodeLockRows.c | 1 +
src/backend/executor/nodeMaterial.c | 5 ++-
src/backend/executor/nodeMemoize.c | 8 +++-
src/backend/executor/nodeMergeAppend.c | 3 ++
src/backend/executor/nodeMergejoin.c | 2 +
src/backend/executor/nodeModifyTable.c | 11 ++++-
src/backend/executor/nodeNestloop.c | 2 +
src/backend/executor/nodeProjectSet.c | 1 +
src/backend/executor/nodeRecursiveunion.c | 24 +++++++++--
src/backend/executor/nodeResult.c | 1 +
src/backend/executor/nodeSamplescan.c | 7 +++-
src/backend/executor/nodeSeqscan.c | 16 +++-----
src/backend/executor/nodeSetOp.c | 6 ++-
src/backend/executor/nodeSort.c | 5 ++-
src/backend/executor/nodeSubqueryscan.c | 1 +
src/backend/executor/nodeTableFuncscan.c | 4 +-
src/backend/executor/nodeTidrangescan.c | 12 ++++--
src/backend/executor/nodeTidscan.c | 8 +++-
src/backend/executor/nodeUnique.c | 1 +
src/backend/executor/nodeWindowAgg.c | 41 ++++++++++++++-----
37 files changed, 248 insertions(+), 120 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..f7f18d3054 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -3010,6 +3010,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if no EvalPlanQualInit() was done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index f154f28902..af22b1676f 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4314,7 +4313,7 @@ ExecEndAgg(AggState *node)
* worker back into shared memory so that it can be picked up by the main
* process to report in EXPLAIN ANALYZE.
*/
- if (node->shared_info && IsParallelWorker())
+ if (node->shared_info != NULL && IsParallelWorker())
{
AggregateInstrumentation *si;
@@ -4327,10 +4326,16 @@ ExecEndAgg(AggState *node)
/* Make sure we have closed any open tuplesorts */
- if (node->sort_in)
+ if (node->sort_in != NULL)
+ {
tuplesort_end(node->sort_in);
- if (node->sort_out)
+ node->sort_in = NULL;
+ }
+ if (node->sort_out != NULL)
+ {
tuplesort_end(node->sort_out);
+ node->sort_out = NULL;
+ }
hashagg_reset_spill_state(node);
@@ -4346,19 +4351,25 @@ ExecEndAgg(AggState *node)
for (setno = 0; setno < numGroupingSets; setno++)
{
- if (pertrans->sortstates[setno])
+ if (pertrans->sortstates[setno] != NULL)
tuplesort_end(pertrans->sortstates[setno]);
}
}
/* And ensure any agg shutdown callbacks have been called */
for (setno = 0; setno < numGroupingSets; setno++)
+ {
ReScanExprContext(node->aggcontexts[setno]);
- if (node->hashcontext)
+ node->aggcontexts[setno] = NULL;
+ }
+ if (node->hashcontext != NULL)
+ {
ReScanExprContext(node->hashcontext);
+ node->hashcontext = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..a2af221e05 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -399,7 +399,10 @@ ExecEndAppend(AppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(appendplans[i]);
+ appendplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..4abb0609a0 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -192,8 +192,8 @@ ExecEndBitmapAnd(BitmapAndState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2db0acfc76..d3f58c22f9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -648,40 +648,59 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
void
ExecEndBitmapHeapScan(BitmapHeapScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * extract information from the node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
+ if (node->tbmiterator != NULL)
+ {
tbm_end_iterate(node->tbmiterator);
- if (node->prefetch_iterator)
+ node->tbmiterator = NULL;
+ }
+ if (node->prefetch_iterator != NULL)
+ {
tbm_end_iterate(node->prefetch_iterator);
- if (node->tbm)
+ node->prefetch_iterator = NULL;
+ }
+ if (node->tbm != NULL)
+ {
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
+ node->tbm = NULL;
+ }
+ if (node->shared_tbmiterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_tbmiterator);
- if (node->shared_prefetch_iterator)
+ node->shared_tbmiterator = NULL;
+ }
+ if (node->shared_prefetch_iterator != NULL)
+ {
tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ node->shared_prefetch_iterator = NULL;
+ }
if (node->vmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->vmbuffer);
+ node->vmbuffer = InvalidBuffer;
+ }
if (node->pvmbuffer != InvalidBuffer)
+ {
ReleaseBuffer(node->pvmbuffer);
+ node->pvmbuffer = InvalidBuffer;
+ }
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 7cf8532bc9..488f11a3ff 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -175,22 +175,21 @@ ExecReScanBitmapIndexScan(BitmapIndexScanState *node)
void
ExecEndBitmapIndexScan(BitmapIndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->biss_RelationDesc;
- indexScanDesc = node->biss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->biss_ScanDesc != NULL)
+ {
+ index_endscan(node->biss_ScanDesc);
+ node->biss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->biss_RelationDesc != NULL)
+ {
+ index_close(node->biss_RelationDesc, NoLock);
+ node->biss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..ace18593aa 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -210,8 +210,8 @@ ExecEndBitmapOr(BitmapOrState *node)
*/
for (i = 0; i < nplans; i++)
{
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
+ ExecEndNode(bitmapplans[i]);
+ bitmapplans[i] = NULL;
}
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 73913ebb18..3aba28285a 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -301,17 +301,20 @@ ExecEndForeignScan(ForeignScanState *node)
EState *estate = node->ss.ps.state;
/* Let the FDW shut down */
- if (plan->operation != CMD_SELECT)
+ if (node->fdwroutine != NULL)
{
- if (estate->es_epq_active == NULL)
- node->fdwroutine->EndDirectModify(node);
+ if (plan->operation != CMD_SELECT)
+ {
+ if (estate->es_epq_active == NULL)
+ node->fdwroutine->EndDirectModify(node);
+ }
+ else
+ node->fdwroutine->EndForeignScan(node);
}
- else
- node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index bb2500a469..1a3c8abdad 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -249,6 +249,7 @@ void
ExecEndGather(GatherState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGather(node);
}
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 7a71a58509..c6fb45fee0 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -289,6 +289,7 @@ void
ExecEndGatherMerge(GatherMergeState *node)
{
ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ outerPlanState(node) = NULL;
ExecShutdownGatherMerge(node);
}
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 8c650f0e46..6dfe5a1d23 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,10 +226,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index e72f0986c2..88ba336882 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,13 +413,11 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* shut down the subplan
*/
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 25a2d78f15..03dd931527 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -861,7 +861,7 @@ ExecEndHashJoin(HashJoinState *node)
/*
* Free hash table
*/
- if (node->hj_HashTable)
+ if (node->hj_HashTable != NULL)
{
ExecHashTableDestroy(node->hj_HashTable);
node->hj_HashTable = NULL;
@@ -871,7 +871,9 @@ ExecEndHashJoin(HashJoinState *node)
* clean up subtrees
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index cd094a190c..28a0e81cb3 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1079,8 +1079,16 @@ ExecEndIncrementalSort(IncrementalSortState *node)
{
SO_printf("ExecEndIncrementalSort: shutting down sort node\n");
- ExecDropSingleTupleTableSlot(node->group_pivot);
- ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ if (node->group_pivot != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->group_pivot);
+ node->group_pivot = NULL;
+ }
+ if (node->transfer_tuple != NULL)
+ {
+ ExecDropSingleTupleTableSlot(node->transfer_tuple);
+ node->transfer_tuple = NULL;
+ }
/*
* Release tuplesort resources.
@@ -1100,6 +1108,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
* Shut down the subplan.
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f1db35665c..1f3843abe9 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -364,15 +364,6 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
void
ExecEndIndexOnlyScan(IndexOnlyScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->ioss_RelationDesc;
- indexScanDesc = node->ioss_ScanDesc;
-
/* Release VM buffer pin, if any. */
if (node->ioss_VMBuffer != InvalidBuffer)
{
@@ -380,13 +371,21 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_VMBuffer = InvalidBuffer;
}
+ /* close the scan (no-op if we didn't start it) */
+ if (node->ioss_ScanDesc != NULL)
+ {
+ index_endscan(node->ioss_ScanDesc);
+ node->ioss_ScanDesc = NULL;
+ }
+
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->ioss_RelationDesc != NULL)
+ {
+ index_close(node->ioss_RelationDesc, NoLock);
+ node->ioss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 14b9c00217..32e1714f15 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -785,22 +785,21 @@ ExecIndexAdvanceArrayKeys(IndexArrayKeyInfo *arrayKeys, int numArrayKeys)
void
ExecEndIndexScan(IndexScanState *node)
{
- Relation indexRelationDesc;
- IndexScanDesc indexScanDesc;
-
- /*
- * extract information from the node
- */
- indexRelationDesc = node->iss_RelationDesc;
- indexScanDesc = node->iss_ScanDesc;
+ /* close the scan (no-op if we didn't start it) */
+ if (node->iss_ScanDesc != NULL)
+ {
+ index_endscan(node->iss_ScanDesc);
+ node->iss_ScanDesc = NULL;
+ }
/*
* close the index relation (no-op if we didn't open it)
*/
- if (indexScanDesc)
- index_endscan(indexScanDesc);
- if (indexRelationDesc)
- index_close(indexRelationDesc, NoLock);
+ if (node->iss_RelationDesc != NULL)
+ {
+ index_close(node->iss_RelationDesc, NoLock);
+ node->iss_RelationDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 5654158e3e..a97bac9f6d 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,6 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..26fbe95c57 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -387,6 +387,7 @@ ExecEndLockRows(LockRowsState *node)
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 753ea28915..03c514900b 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -243,13 +243,16 @@ ExecEndMaterial(MaterialState *node)
* Release tuplestore resources
*/
if (node->tuplestorestate != NULL)
+ {
tuplestore_end(node->tuplestorestate);
- node->tuplestorestate = NULL;
+ node->tuplestorestate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 1085b3c79d..9b13e2e552 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1062,6 +1062,7 @@ ExecEndMemoize(MemoizeState *node)
{
#ifdef USE_ASSERT_CHECKING
/* Validate the memory accounting code is correct in assert builds. */
+ if (node->hashtable != NULL)
{
int count;
uint64 mem = 0;
@@ -1108,12 +1109,17 @@ ExecEndMemoize(MemoizeState *node)
}
/* Remove the cache context */
- MemoryContextDelete(node->tableContext);
+ if (node->tableContext != NULL)
+ {
+ MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..0a42a04b19 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -333,7 +333,10 @@ ExecEndMergeAppend(MergeAppendState *node)
* shut down each of the subscans
*/
for (i = 0; i < nplans; i++)
+ {
ExecEndNode(mergeplans[i]);
+ mergeplans[i] = NULL;
+ }
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 3cdab77dfc..4b181621f9 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1647,7 +1647,9 @@ ExecEndMergeJoin(MergeJoinState *node)
* shut down the subplans
*/
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index b16fbe9e22..bc82fb033a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4447,7 +4447,9 @@ ExecEndModifyTable(ModifyTableState *node)
for (j = 0; j < resultRelInfo->ri_NumSlotsInitialized; j++)
{
ExecDropSingleTupleTableSlot(resultRelInfo->ri_Slots[j]);
+ resultRelInfo->ri_Slots[j] = NULL;
ExecDropSingleTupleTableSlot(resultRelInfo->ri_PlanSlots[j]);
+ resultRelInfo->ri_PlanSlots[j] = NULL;
}
}
@@ -4455,12 +4457,16 @@ ExecEndModifyTable(ModifyTableState *node)
* Close all the partitioned tables, leaf partitions, and their indices
* and release the slot used for tuple routing, if set.
*/
- if (node->mt_partition_tuple_routing)
+ if (node->mt_partition_tuple_routing != NULL)
{
ExecCleanupTupleRouting(node, node->mt_partition_tuple_routing);
+ node->mt_partition_tuple_routing = NULL;
- if (node->mt_root_tuple_slot)
+ if (node->mt_root_tuple_slot != NULL)
+ {
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
+ node->mt_root_tuple_slot = NULL;
+ }
}
/*
@@ -4472,6 +4478,7 @@ ExecEndModifyTable(ModifyTableState *node)
* shut down subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index ebd1406843..1211d871ea 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -368,7 +368,9 @@ ExecEndNestLoop(NestLoopState *node)
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index aee26d3813..514669558c 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -332,6 +332,7 @@ ExecEndProjectSet(ProjectSetState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index 3207643156..08637a64db 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -272,20 +272,36 @@ void
ExecEndRecursiveUnion(RecursiveUnionState *node)
{
/* Release tuplestores */
- tuplestore_end(node->working_table);
- tuplestore_end(node->intermediate_table);
+ if (node->working_table != NULL)
+ {
+ tuplestore_end(node->working_table);
+ node->working_table = NULL;
+ }
+ if (node->intermediate_table != NULL)
+ {
+ tuplestore_end(node->intermediate_table);
+ node->intermediate_table = NULL;
+ }
/* free subsidiary stuff including hashtable */
- if (node->tempContext)
+ if (node->tempContext != NULL)
+ {
MemoryContextDelete(node->tempContext);
- if (node->tableContext)
+ node->tempContext = NULL;
+ }
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
/*
* close down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
ExecEndNode(innerPlanState(node));
+ innerPlanState(node) = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index e9f5732f33..f15902e840 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -244,6 +244,7 @@ ExecEndResult(ResultState *node)
* shut down subplans
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
void
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 41c1ea37ad..a6813559e6 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -185,14 +185,17 @@ ExecEndSampleScan(SampleScanState *node)
/*
* Tell sampling function that we finished the scan.
*/
- if (node->tsmroutine->EndSampleScan)
+ if (node->tsmroutine != NULL && node->tsmroutine->EndSampleScan)
node->tsmroutine->EndSampleScan(node);
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
if (node->ss.ss_currentScanDesc)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 49a5933aff..911266da07 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -183,18 +183,14 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
void
ExecEndSeqScan(SeqScanState *node)
{
- TableScanDesc scanDesc;
-
- /*
- * get information from node
- */
- scanDesc = node->ss.ss_currentScanDesc;
-
/*
- * close heap scan
+ * close heap scan (no-op if we didn't start it)
*/
- if (scanDesc != NULL)
- table_endscan(scanDesc);
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 5a84969cf8..92bd2da8e0 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -583,10 +583,14 @@ void
ExecEndSetOp(SetOpState *node)
{
/* free subsidiary stuff including hashtable */
- if (node->tableContext)
+ if (node->tableContext != NULL)
+ {
MemoryContextDelete(node->tableContext);
+ node->tableContext = NULL;
+ }
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index eea7f2ae15..c8a35b64a8 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -307,13 +307,16 @@ ExecEndSort(SortState *node)
* Release tuplesort resources
*/
if (node->tuplesortstate != NULL)
+ {
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
- node->tuplesortstate = NULL;
+ node->tuplesortstate = NULL;
+ }
/*
* shut down the subplan
*/
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 1ee6295660..91d7ae82ce 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -171,6 +171,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
* close down subquery
*/
ExecEndNode(node->subplan);
+ node->subplan = NULL;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index a60dcd4943..80ed4b26a8 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -217,8 +217,10 @@ ExecEndTableFuncScan(TableFuncScanState *node)
* Release tuplestore resources
*/
if (node->tupstore != NULL)
+ {
tuplestore_end(node->tupstore);
- node->tupstore = NULL;
+ node->tupstore = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 6f97c35daa..d6b4ed2e42 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -327,10 +327,14 @@ ExecReScanTidRangeScan(TidRangeScanState *node)
void
ExecEndTidRangeScan(TidRangeScanState *node)
{
- TableScanDesc scan = node->ss.ss_currentScanDesc;
-
- if (scan != NULL)
- table_endscan(scan);
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
+ table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 15055077d0..74ec6afdcc 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -470,8 +470,14 @@ ExecReScanTidScan(TidScanState *node)
void
ExecEndTidScan(TidScanState *node)
{
- if (node->ss.ss_currentScanDesc)
+ /*
+ * close heap scan (no-op if we didn't start it)
+ */
+ if (node->ss.ss_currentScanDesc != NULL)
+ {
table_endscan(node->ss.ss_currentScanDesc);
+ node->ss.ss_currentScanDesc = NULL;
+ }
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 01f951197c..13c556326a 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -169,6 +169,7 @@ void
ExecEndUnique(UniqueState *node)
{
ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 3258305f57..0253c47448 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -1351,11 +1351,14 @@ release_partition(WindowAggState *winstate)
* any aggregate temp data). We don't rely on retail pfree because some
* aggregates might have allocated data we don't have direct pointers to.
*/
- MemoryContextReset(winstate->partcontext);
- MemoryContextReset(winstate->aggcontext);
+ if (winstate->partcontext != NULL)
+ MemoryContextReset(winstate->partcontext);
+ if (winstate->aggcontext != NULL)
+ MemoryContextReset(winstate->aggcontext);
for (i = 0; i < winstate->numaggs; i++)
{
- if (winstate->peragg[i].aggcontext != winstate->aggcontext)
+ if (winstate->peragg[i].aggcontext != NULL &&
+ winstate->peragg[i].aggcontext != winstate->aggcontext)
MemoryContextReset(winstate->peragg[i].aggcontext);
}
@@ -2681,24 +2684,40 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
for (i = 0; i < node->numaggs; i++)
{
- if (node->peragg[i].aggcontext != node->aggcontext)
+ if (node->peragg[i].aggcontext != NULL &&
+ node->peragg[i].aggcontext != node->aggcontext)
MemoryContextDelete(node->peragg[i].aggcontext);
}
- MemoryContextDelete(node->partcontext);
- MemoryContextDelete(node->aggcontext);
+ if (node->partcontext != NULL)
+ {
+ MemoryContextDelete(node->partcontext);
+ node->partcontext = NULL;
+ }
+ if (node->aggcontext != NULL)
+ {
+ MemoryContextDelete(node->aggcontext);
+ node->aggcontext = NULL;
+ }
- pfree(node->perfunc);
- pfree(node->peragg);
+ if (node->perfunc != NULL)
+ {
+ pfree(node->perfunc);
+ node->perfunc = NULL;
+ }
+ if (node->peragg != NULL)
+ {
+ pfree(node->peragg);
+ node->peragg = NULL;
+ }
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ ExecEndNode(outerPlanState(node));
+ outerPlanState(node) = NULL;
}
/* -----------------
--
2.35.3
[application/octet-stream] v49-0003-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch (49.3K, 9-v49-0003-Adjustments-to-allow-ExecutorStart-to-sometimes-.patch)
download | inline diff:
From fa5e9ab0316bd57f0248982729f906ddfbdca80e Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Wed, 6 Sep 2023 17:53:46 +0900
Subject: [PATCH v49 3/8] Adjustments to allow ExecutorStart() to sometimes
fail
Upon passing a plan tree from a CachedPlan to the executor, there's a
possibility that ExecutorStart() might return an incompletely set up
planstate tree. This can happen if the CachedPlan undergoes invalidation
during the ExecInitNode() initialization process. In such cases, the
execution should be reattempted using a fresh CachedPlan. Also, any
partially initialized EState must be cleaned up by invoking both
ExecutorEnd() and FreeExecutorState().
ExecutorStart() (and ExecutorStart_hook()) now return a Boolean telling
the caller if the plan initialization failed.
For the replan loop in that context, it makes more sense to have
ExecutorStart() either in the same scope or closer to where
GetCachedPlan() is invoked. So this commit modifies the following
sites:
* The ExecutorStart() call in ExplainOnePlan() is moved into a new
function ExplainQueryDesc() along with CreateQueryDesc(). Callers
of ExplainOnePlan() should now call the new function first.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and a new memory
context for those. One unintended consequence is that
CommandCounterIncrement() between queries in the PORTAL_MULTI_QUERY
case is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still works because the Snapshot registered
in QueryDesc/EState is updated to account for the CCI().
This commit also adds a new flag to EState called es_canceled that
complements es_finished to denote the new scenario where
ExecutorStart() returns with a partially setup planstate tree. Also,
to reset the AFTER trigger state that would have been set up in the
ExecutorStart(), this adds a new function AfterTriggerCancelQuery()
which is called from ExecutorEnd() (not ExecutorFinish()) when
es_canceled is true.
Note that this commit by itself doesn't make any functional change,
because the CachedPlan is not passed into the executor yet.
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
src/backend/commands/copyto.c | 5 +-
src/backend/commands/createas.c | 9 +-
src/backend/commands/explain.c | 145 +++++---
src/backend/commands/extension.c | 6 +-
src/backend/commands/matview.c | 9 +-
src/backend/commands/portalcmds.c | 6 +-
src/backend/commands/prepare.c | 31 +-
src/backend/commands/trigger.c | 13 +
src/backend/executor/execMain.c | 44 ++-
src/backend/executor/execParallel.c | 6 +-
src/backend/executor/execUtils.c | 1 +
src/backend/executor/functions.c | 7 +-
src/backend/executor/spi.c | 48 ++-
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 346 +++++++++---------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 7 +-
src/include/commands/trigger.h | 1 +
src/include/executor/executor.h | 6 +-
src/include/nodes/execnodes.h | 3 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/portal.h | 2 +
24 files changed, 466 insertions(+), 282 deletions(-)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 6c63acf989..6325eeaf46 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -326,7 +326,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -979,13 +979,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -1008,6 +1010,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 0929ad929a..699060fd55 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -568,8 +568,11 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 18b07c0200..4a950c03ff 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -329,8 +329,13 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c698e54fec..7b57cd02aa 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,90 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to have been invalidated after
+ * calling ExecutorStart().
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +603,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
-
- Assert(plannedstmt->commandType != CMD_UTILITY);
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +621,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, NULL, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
@@ -4895,6 +4927,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index b287a2e84c..127d2a3b0a 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -802,7 +802,11 @@ execute_sql_string(const char *sql)
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 22b8b820c3..7083fb2350 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -412,8 +412,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..a1ee5c0acd 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,11 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here,
+ * because there's no CachedPlan.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..f8d0b0ee25 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal has a cached plan and
+ * it's found to be invalidated during the initialization of its plan
+ * trees, the plan must be regenerated.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -639,8 +647,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
+ into, es, paramLI, queryEnv);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 52177759ab..dd139432b9 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -5009,6 +5009,19 @@ AfterTriggerBeginQuery(void)
afterTriggers.query_depth++;
}
+/* ----------
+ * AfterTriggerCancelQuery()
+ *
+ * Called from ExecutorEnd() if the query execution was canceled.
+ * ----------
+ */
+void
+AfterTriggerCancelQuery(void)
+{
+ /* Set to a value denoting that no query is active. */
+ afterTriggers.query_depth = -1;
+}
+
/* ----------
* AfterTriggerEndQuery()
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index de7bf7ca67..5755336abd 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,14 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- (void) InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
+
+ /* Mark execution as canceled if plan won't be executed. */
+ estate->es_canceled = !plan_valid;
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -325,6 +338,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
estate = queryDesc->estate;
Assert(estate != NULL);
+ Assert(!estate->es_canceled);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -429,7 +443,7 @@ standard_ExecutorFinish(QueryDesc *queryDesc)
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/* This should be run once and only once per Executor instance */
- Assert(!estate->es_finished);
+ Assert(!estate->es_finished && !estate->es_canceled);
/* Switch into per-query memory context */
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
@@ -488,11 +502,11 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
Assert(estate != NULL);
/*
- * Check that ExecutorFinish was called, unless in EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
- Assert(estate->es_finished ||
+ Assert(estate->es_finished || estate->es_canceled ||
(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
/*
@@ -506,6 +520,14 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
UnregisterSnapshot(estate->es_snapshot);
UnregisterSnapshot(estate->es_crosscheck_snapshot);
+ /*
+ * Cancel trigger execution too if the query execution was canceled.
+ */
+ if (estate->es_canceled &&
+ !(estate->es_top_eflags &
+ (EXEC_FLAG_SKIP_TRIGGERS | EXEC_FLAG_EXPLAIN_ONLY)))
+ AfterTriggerCancelQuery();
+
/*
* Must switch out of context before destroying it
*/
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 457ee46faf..13d2820a41 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1437,7 +1437,11 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 16704c0c2f..f0f5740c26 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -151,6 +151,7 @@ CreateExecutorState(void)
estate->es_top_eflags = 0;
estate->es_instrument = 0;
estate->es_finished = false;
+ estate->es_canceled = false;
estate->es_exprcontexts = NIL;
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 66636b05a5..27565a8b78 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -863,7 +863,12 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /*
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
+ */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 892b2853ed..60e2632cd2 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2675,8 +2687,23 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
FreeQueryDesc(qdesc);
}
else
@@ -2851,10 +2878,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2898,14 +2924,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e415cf1f34..70e7a023d5 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1231,7 +1231,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1736,6 +1741,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2023,9 +2029,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 4ef349df8b..fcf9925ed4 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -118,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, NULL, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -428,19 +343,21 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -450,15 +367,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -474,6 +389,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -491,8 +408,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
NULL,
@@ -504,30 +421,51 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
/*
- * Call ExecutorStart to prepare the plan for execution
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
*/
- ExecutorStart(queryDesc, myeflags);
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -539,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -584,7 +499,82 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ NULL,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -597,19 +587,20 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
@@ -1196,7 +1187,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1217,9 +1208,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1236,33 +1228,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1271,26 +1256,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1345,12 +1343,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index f9525fb572..054132823c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
@@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 430e3ca7dd..d4f7c29301 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -257,6 +257,7 @@ extern void ExecASTruncateTriggers(EState *estate,
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
+extern void AfterTriggerCancelQuery(void);
extern void AfterTriggerEndQuery(EState *estate);
extern void AfterTriggerFireDeferred(void);
extern void AfterTriggerEndXact(bool isCommit);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3b33b38196..4f183ec6cd 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -73,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -198,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 08670bc5ed..cad2329ac9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -669,6 +669,9 @@ typedef struct EState
int es_top_eflags; /* eflags passed to ExecutorStart */
int es_instrument; /* OR of InstrumentOption flags */
bool es_finished; /* true when ExecutorFinish is done */
+ bool es_canceled; /* true when execution was canceled
+ * upon encountering that plan was invalided
+ * during ExecInitNode() */
List *es_exprcontexts; /* List of ExprContexts within EState */
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index 8b4471cbe5..513e3c388d 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2023-12-06 18:52 Robert Haas <[email protected]>
parent: Amit Langote <[email protected]>
2 siblings, 0 replies; 31+ messages in thread
From: Robert Haas @ 2023-12-06 18:52 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
Reviewing 0001:
Perhaps ExecEndCteScan needs an adjustment. What if node->leader was never set?
Other than that, I think this is in good shape. Maybe there are other
things we'd want to adjust here, or maybe there aren't, but there
doesn't seem to be any good reason to bundle more changes into the
same patch.
Reviewing 0002 and beyond:
I think it's good that you have tried to divide up a big change into
little pieces, but I'm finding the result difficult to understand. It
doesn't really seem like each patch stands on its own. I keep flipping
between patches to try to understand why other patches are doing
things, which kind of defeats the purpose of splitting stuff up. For
example, 0002 adds a NodeTag field to QueryDesc, but it doesn't even
seem to initialize that field, let alone use it for anything. It adds
a CachedPlan pointer to QueryDesc too, and adapts CreateQueryDesc to
allow one as an argument, but none of the callers actually pass
anything. I suspect that that the first change (adding a NodeTag)
field is a bug, and that the second one is intentional, but it's hard
to tell without flipping through all of the other patches to see how
they build on what 0002 does. And even when something isn't a bug,
it's also hard to tell whether it's the right design, again because
you can't consider each patch in isolation. Ideally, splitting a patch
set should bring related changes together in a single patch and push
unrelated changes apart into different patches, but I don't really see
this particular split having that effect.
There is a chicken and egg problem here, to be fair. If we add code
that can make plan initialization fail without teaching the planner to
cope with failures, then we have broken the server, and if we do the
reverse, then we have a bunch of dead code that we can't test. Neither
is very satisfactory. But I still hope there's some better division
possible than what you have here currently. For instance, I wonder if
it would be possible to add all the stuff to cope with plan
initialization failing and then have a test patch that makes
initialization randomly fail with some probability (or maybe you can
even cause failures at specific points). Then you could test that
infrastructure by running the regression tests in a loop with various
values of the relevant setting.
Another overall comment that I have is that it doesn't feel like
there's enough high-level explanation of the design. I don't know how
much of that should go in comments vs. commit messages vs. a README
that accompanies the patch set vs. whatever else, and I strongly
suspect that some of the stuff that seems confusing now is actually
stuff that at one point I understood and have just forgotten about.
But rediscovering it shouldn't be quite so hard. For example, consider
the question "why are we storing the CachedPlan in the QueryDesc?" I
eventually figured out that it's so that ExecPlanStillValid can call
CachedPlanStillValid which can then consult the cached plan's is_valid
flag. But is that the only access to the CachedPlan that we ever
expect to occur via the QueryDesc? If not, what else is allowable? If
so, why not just store a Boolean in the QueryDesc and arrange for the
plancache to be able to flip it when invalidating? I'm not saying
that's a better design -- I'm saying that it looks hard to understand
your thought process from the patch set. And also, you know, assuming
the current design is correct, could there be some way of dividing up
the patch set so that this one change, where we add the CachedPlan to
the QueryDesc, isn't so spread out across the whole series?
Some more detailed review comments below. This isn't really a full
review because I don't understand the patches well enough for that,
but it's some stuff I noticed.
In 0002:
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
Typo.
+ /*
+ * All the necessary locks must already have been taken when
+ * initializing the parent's copy of subplanstate, so the CachedPlan,
+ * if any, should not have become invalid during ExecInitNode().
+ */
+ Assert(ExecPlanStillValid(rcestate));
This -- and the other similar instance -- feel very uncomfortable.
There's a lot of action at a distance here. If this assertion ever
failed, how would anyone ever figure out what went wrong? You wouldn't
for example know which object got invalidated, presumably
corresponding to a lock that you failed to take. Unless the problem
were easily reproducible in a test environment, trying to guess what
happened might be pretty awful; imagine seeing this assertion failure
in a customer log file and trying to back-track to the find the
underlying bug. A further problem is that what would actually happen
is you *wouldn't* see this in the customer log file, because
assertions wouldn't be enabled, so you'd just see queries occasionally
returning wrong answers, I guess? Or crashing in some other random
part of the code? Which seems even worse. At a minimum I think this
should be upgraded to a test-and-elog, and maybe there's some value in
trying to think of what should get printed by that elog to facilitate
proper debugging, if it happens.
In 0003:
+ *
+ * OK to ignore the return value; plan can't become invalid,
+ * because there's no CachedPlan.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
This also feels awkward, for similar reasons. Sure, it shouldn't
return false, but also, if it did, you'd just blindly continue. Maybe
there should be test-and-elog here too. Or maybe this is an indication
that we need less action at a distance. Like, if ExecutorStart took
the CachedPlan as an argument instead of feeding it through the
QueryDesc, then you could document that ExecutorStart returns true if
that value is passed as NULL and true or false otherwise. Here,
whether ExecutorStart can return true or false depends on the contents
of the queryDesc ... which, granted, in this case is just built a line
or two before anyway, but if you just passed to to ExecutorStart then
you wouldn't need to feed it through the QueryDesc, it seems to me.
Even better, maybe there should be ExecutorStart() that continues
returning void and ExecutorStartExtended() that takes a cached plan as
an additional argument and returns a bool.
/*
- * Check that ExecutorFinish was called, unless in
EXPLAIN-only mode. This
- * Assert is needed because ExecutorFinish is new as of 9.1, and callers
- * might forget to call it.
+ * Check that ExecutorFinish was called, unless in
EXPLAIN-only mode or if
+ * execution was canceled. This Assert is needed because
ExecutorFinish is
+ * new as of 9.1, and callers might forget to call it.
*/
Maybe we could drop the second sentence at this point.
In 0005:
+ * XXX Maybe we should we skip calling
ExecCheckPermissions from
+ * InitPlan in a parallel worker.
Why? If the thinking is to save overhead, then perhaps try to assess
the overhead. If the thinking is that we don't want it to fail
spuriously, then we have to weight that against the (security) risk of
succeeding spuriously.
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
I don't particularly enjoy seeing this comment cut and pasted into
some new place. Especially the tongue-in-cheek parenthetical part.
Better to refer to the original comment or something instead of
cut-and-pasting. Also, why is it appropriate to pass orstronger = true
here? Don't we expect the *exact* lock mode that we have planned to be
held, and isn't it a sure sign of a bug if it isn't? Maybe orstronger
should just be ripped out here (and the comment could then go away
too).
In 0006:
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
The first sentence of this comment makes this sound like a list of
integers, the RTIs of all partitioned tables that are scanned. The
second sentence makes it sound like a list of bitmapsets, but what
does it mean to take about each partition tree covered by this Append?
This is far from a complete review but I'm running out of steam for
today. I hope that it's at least somewhat useful.
...Robert
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2024-01-05 10:46 vignesh C <[email protected]>
parent: Amit Langote <[email protected]>
2 siblings, 1 reply; 31+ messages in thread
From: vignesh C @ 2024-01-05 10:46 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
On Mon, 20 Nov 2023 at 10:00, Amit Langote <[email protected]> wrote:
>
> On Thu, Sep 28, 2023 at 5:26 PM Amit Langote <[email protected]> wrote:
> > On Tue, Sep 26, 2023 at 10:06 PM Amit Langote <[email protected]> wrote:
> > > After sleeping on this, I think we do need the checks after all the
> > > ExecInitNode() calls too, because we have many instances of the code
> > > like the following one:
> > >
> > > outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
> > > tupDesc = ExecGetResultType(outerPlanState(gatherstate));
> > > <some code that dereferences outDesc>
> > >
> > > If outerNode is a SeqScan and ExecInitSeqScan() returned early because
> > > ExecOpenScanRelation() detected that plan was invalidated, then
> > > tupDesc would be NULL in this case, causing the code to crash.
> > >
> > > Now one might say that perhaps we should only add the
> > > is-CachedPlan-valid test in the instances where there is an actual
> > > risk of such misbehavior, but that could lead to confusion, now or
> > > later. It seems better to add them after every ExecInitNode() call
> > > while we're inventing the notion, because doing so relieves the
> > > authors of future enhancements of the ExecInit*() routines from
> > > worrying about any of this.
> > >
> > > Attached 0003 should show how that turned out.
> > >
> > > Updated 0002 as mentioned in the previous reply -- setting pointers to
> > > NULL after freeing them more consistently across various ExecEnd*()
> > > routines and using the `if (pointer != NULL)` style over the `if
> > > (pointer)` more consistently.
> > >
> > > Updated 0001's commit message to remove the mention of its relation to
> > > any future commits. I intend to push it tomorrow.
> >
> > Pushed that one. Here are the rebased patches.
> >
> > 0001 seems ready to me, but I'll wait a couple more days for others to
> > weigh in. Just to highlight a kind of change that others may have
> > differing opinions on, consider this hunk from the patch:
> >
> > - MemoryContextDelete(node->aggcontext);
> > + if (node->aggcontext != NULL)
> > + {
> > + MemoryContextDelete(node->aggcontext);
> > + node->aggcontext = NULL;
> > + }
> > ...
> > + ExecEndNode(outerPlanState(node));
> > + outerPlanState(node) = NULL;
> >
> > So the patch wants to enhance the consistency of setting the pointer
> > to NULL after freeing part. Robert mentioned his preference for doing
> > it in the patch, which I agree with.
>
> Rebased.
There is a leak reported at [1], details for the same is available at [2]:
diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/select_views.out
/tmp/cirrus-ci-build/build/testrun/regress-running/regress/results/select_views.out
--- /tmp/cirrus-ci-build/src/test/regress/expected/select_views.out
2023-12-19 23:00:04.677385000 +0000
+++ /tmp/cirrus-ci-build/build/testrun/regress-running/regress/results/select_views.out
2023-12-19 23:06:26.870259000 +0000
@@ -1288,6 +1288,7 @@
(102, '2011-10-12', 120),
(102, '2011-10-28', 200),
(103, '2011-10-15', 480);
+WARNING: resource was not closed: relation "customer_pkey"
CREATE VIEW my_property_normal AS
SELECT * FROM customer WHERE name = current_user;
CREATE VIEW my_property_secure WITH (security_barrier) A
[1] - https://cirrus-ci.com/task/6494009196019712
[2] - https://api.cirrus-ci.com/v1/artifact/task/6494009196019712/testrun/build/testrun/regress-running/re...
Regards,
Vingesh
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2024-03-31 05:03 Andrey M. Borodin <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Andrey M. Borodin @ 2024-03-31 05:03 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Amit Langote <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
> On 6 Dec 2023, at 23:52, Robert Haas <[email protected]> wrote:
>
> I hope that it's at least somewhat useful.
>
> On 5 Jan 2024, at 15:46, vignesh C <[email protected]> wrote:
>
> There is a leak reported
Hi Amit,
this is a kind reminder that some feedback on your patch[0] is waiting for your reply.
Thank you for your work!
Best regards, Andrey Borodin.
[0] https://commitfest.postgresql.org/47/3478/
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2024-04-08 08:39 Amit Langote <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Amit Langote @ 2024-04-08 08:39 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: vignesh C <[email protected]>; Robert Haas <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
Hi Andrey,
On Sun, Mar 31, 2024 at 2:03 PM Andrey M. Borodin <[email protected]> wrote:
> > On 6 Dec 2023, at 23:52, Robert Haas <[email protected]> wrote:
> >
> > I hope that it's at least somewhat useful.
>
> > On 5 Jan 2024, at 15:46, vignesh C <[email protected]> wrote:
> >
> > There is a leak reported
>
> Hi Amit,
>
> this is a kind reminder that some feedback on your patch[0] is waiting for your reply.
> Thank you for your work!
Thanks for moving this to the next CF.
My apologies (especially to Robert) for not replying on this thread
for a long time.
I plan to start working on this soon.
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: generic plans and "initial" pruning
@ 2024-06-19 17:09 Alvaro Herrera <[email protected]>
parent: Amit Langote <[email protected]>
2 siblings, 0 replies; 31+ messages in thread
From: Alvaro Herrera @ 2024-06-19 17:09 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Thom Brown <[email protected]>; Tom Lane <[email protected]>
I had occasion to run the same benchmark you described in the initial
email in this thread. To do so I applied patch series v49 on top of
07cb29737a4e, which is just one that happened to have the same date as
v49.
I then used a script like this (against a server having
plan_cache_mode=force_generic_mode)
for numparts in 0 1 2 4 8 16 32 48 64 80 81 96 127 128 160 200 256 257 288 300 384 512 1024 1536 2048; do
pgbench testdb -i --partitions=$numparts 2>/dev/null
echo -ne "$numparts\t"
pgbench -n testdb -S -T30 -Mprepared | grep "^tps" | sed -e 's/^tps = \([0-9.]*\) .*/\1/'
done
and did the same with the commit mentioned above (that is, unpatched).
I got this table as result
partitions │ patched │ 07cb29737a
────────────┼──────────────┼──────────────
0 │ 65632.090431 │ 68967.712741
1 │ 68096.641831 │ 65356.587223
2 │ 59456.507575 │ 60884.679464
4 │ 62097.426 │ 59698.747104
8 │ 58044.311175 │ 57817.104562
16 │ 59741.926563 │ 52549.916262
32 │ 59261.693449 │ 44815.317215
48 │ 59047.125629 │ 38362.123652
64 │ 59748.738797 │ 34051.158525
80 │ 59276.839183 │ 32026.135076
81 │ 62318.572932 │ 30418.122933
96 │ 59678.857163 │ 28478.113651
127 │ 58761.960028 │ 24272.303742
128 │ 59934.268306 │ 24275.214593
160 │ 56688.790899 │ 21119.043564
200 │ 56323.188599 │ 18111.212849
256 │ 55915.22466 │ 14753.953709
257 │ 57810.530461 │ 15093.497575
288 │ 56874.780092 │ 13873.332162
300 │ 57222.056549 │ 13463.768946
384 │ 54073.77295 │ 11183.558339
512 │ 37503.766847 │ 8114.32532
1024 │ 42746.866448 │ 4468.41359
1536 │ 39500.58411 │ 3049.984599
2048 │ 36988.519486 │ 2269.362006
where already at 16 partitions we can see that things are going downhill
with the unpatched code. (However, what happens when the table is not
partitioned looks a bit funny.)
I hope we can get this new executor code in 18.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"La primera ley de las demostraciones en vivo es: no trate de usar el sistema.
Escriba un guión que no toque nada para no causar daños." (Jakob Nielsen)
^ permalink raw reply [nested|flat] 31+ messages in thread
end of thread, other threads:[~2024-06-19 17:09 UTC | newest]
Thread overview: 31+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-04-05 23:23 Re: generic plans and "initial" pruning Amit Langote <[email protected]>
2023-06-08 14:23 ` Amit Langote <[email protected]>
2023-07-03 13:27 ` Daniel Gustafsson <[email protected]>
2023-07-06 14:29 ` Amit Langote <[email protected]>
2023-07-13 12:58 ` Amit Langote <[email protected]>
2023-07-17 16:32 ` Thom Brown <[email protected]>
2023-07-18 07:26 ` Amit Langote <[email protected]>
2023-07-18 08:36 ` Thom Brown <[email protected]>
2023-08-02 13:39 ` Amit Langote <[email protected]>
2023-08-03 08:37 ` Amit Langote <[email protected]>
2023-08-07 15:36 ` Robert Haas <[email protected]>
2023-08-07 15:44 ` Tom Lane <[email protected]>
2023-08-07 16:25 ` Robert Haas <[email protected]>
2023-08-08 14:32 ` Amit Langote <[email protected]>
2023-08-08 16:05 ` Robert Haas <[email protected]>
2023-08-11 05:31 ` Amit Langote <[email protected]>
2023-08-11 13:50 ` Amit Langote <[email protected]>
2023-08-28 13:43 ` Robert Haas <[email protected]>
2023-09-05 07:13 ` Amit Langote <[email protected]>
2023-09-05 14:41 ` Robert Haas <[email protected]>
2023-09-06 09:12 ` Amit Langote <[email protected]>
2023-09-06 14:20 ` Robert Haas <[email protected]>
2023-09-25 12:57 ` Amit Langote <[email protected]>
2023-09-26 13:06 ` Amit Langote <[email protected]>
2023-09-28 08:26 ` Amit Langote <[email protected]>
2023-11-20 04:29 ` Amit Langote <[email protected]>
2023-12-06 18:52 ` Robert Haas <[email protected]>
2024-01-05 10:46 ` vignesh C <[email protected]>
2024-03-31 05:03 ` Andrey M. Borodin <[email protected]>
2024-04-08 08:39 ` Amit Langote <[email protected]>
2024-06-19 17:09 ` Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox