public inbox for [email protected]
help / color / mirror / Atom feedFrom: Amit Langote <[email protected]>
To: Thom Brown <[email protected]>
Cc: Daniel Gustafsson <[email protected]>
Cc: Tom Lane <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: David Rowley <[email protected]>
Cc: Jacob Champion <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Robert Haas <[email protected]>
Subject: Re: generic plans and "initial" pruning
Date: Thu, 3 Aug 2023 17:37:39 +0900
Message-ID: <CA+HiwqE=qxN5C-oN5vguBZOZGyDRAMV2EW1pO_hObcpf6X5QwQ@mail.gmail.com> (raw)
In-Reply-To: <CA+HiwqEP3j25702EeergM7o8GqC79Dx-3gHKnvfa8oRJiXBDgA@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<[email protected]>
<CA+HiwqFhQ8tLMLQAWYRWiBQUrWSLM8qhVzJ4B5jWr=orUXfSyA@mail.gmail.com>
<[email protected]>
<CA+HiwqGFm_5aUqPnt=WCarkJ2ZU6F8kD8pFeGurHP+NZWS8KQw@mail.gmail.com>
<CA+HiwqEDr9m3NGrmiOgatCnRPwD95=MHgWQdwvnoMyQd3k9-Yw@mail.gmail.com>
<CA+HiwqGTrQ=ywAmB2zP81jcENZh1vLuyJaC2-xhWvBsnXWgZYQ@mail.gmail.com>
<CA+HiwqGe0W2C+SrKcxrk-r4JjO0sCfL581p1M2bzr_LSrzGn+g@mail.gmail.com>
<[email protected]>
<CA+HiwqGBumxzWctnUy33dHy2uGCtfmcqKyw4FONJNyitJvujWw@mail.gmail.com>
<CA+HiwqHmdH2bcUtBGncwB7iJ9N0VTkUo4YPYFNtJL_f3kkau=g@mail.gmail.com>
<CA+HiwqHKTxfaYc=e4mVOv0iDm3vVK56WOBCddzYdXKaWQqniww@mail.gmail.com>
<CA+HiwqHQ1PM+HXoEdvutj0huhu2cfmuPa8wtctor0NNADzZVvA@mail.gmail.com>
<CA+HiwqH=cbBocfSmyjd_N7ZceZ3RtXuQ=rNkAfdn+RwqMGY9fQ@mail.gmail.com>
<CA+HiwqHoZSM4A0HKoTERmp=_stQjpjmomgg=rCf_4x4qCpxbZA@mail.gmail.com>
<[email protected]>
<CA+HiwqFfC7ANtb+HAHYuR4wnwYbQdbK5B0ee0fjtNwTt+TOdwg@mail.gmail.com>
<CA+HiwqH3jY-W=bekWxFF=B+9tpS42_1sJGsre1Ks0ueQjhta2Q@mail.gmail.com>
<CA+HiwqEAHH=_PVG87rSHhQxmbHQ1dxSd58BVg=dHHfsgCeQFHw@mail.gmail.com>
<[email protected]>
<CA+HiwqHrkhNe=EUixymT0Nynp78Dnaqnf5qQnCowJd3ZSzXvFg@mail.gmail.com>
<CA+HiwqEDbf=+s73hAF0PigWORRx+YWwbCQtuuWtHzc3ko_DGpw@mail.gmail.com>
<CAA-aLv5EDpYBaZrPjE_kkaoERQmAPHO=fm-FwDsw3xJG5gb8Lg@mail.gmail.com>
<CA+HiwqHBbptyxjQx7964DjitA8FVNs1MN=uwrzRy=oOD0Hy3ag@mail.gmail.com>
<CA+HiwqEP3j25702EeergM7o8GqC79Dx-3gHKnvfa8oRJiXBDgA@mail.gmail.com>
On Wed, Aug 2, 2023 at 10:39 PM Amit Langote <[email protected]> wrote:
> Having extracted the ExecEndNode() change, I'm also starting to feel
> inclined to extract a couple of other bits from the main patch as
> separate patches, such as moving the ExecutorStart() call from
> PortalRun() to PortalStart() for the multi-query portals. I'll do
> that in the next version.
Here's a patch set where the refactoring to move the ExecutorStart()
calls to be closer to GetCachedPlan() (for the call sites that use a
CachedPlan) is extracted into a separate patch, 0002. Its commit
message notes an aspect of this refactoring that I feel a bit nervous
about -- needing to also move the CommandCounterIncrement() call from
the loop in PortalRunMulti() to PortalStart() which now does
ExecutorStart() for the PORTAL_MULTI_QUERY case.
--
Thanks, Amit Langote
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] v44-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch (3.7K, 2-v44-0004-Set-inFromCl-to-false-in-child-table-RTEs.patch)
download | inline diff:
From 26df10ea36b2089d59129b066d3dfaedb3aa5e0c Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:43 +0900
Subject: [PATCH v44 4/6] Set inFromCl to false in child table RTEs
This is to allow the executor be able to distinguish tables that are
directly mentioned in the query from those that get added to the
query during planning. A subsequent commit will teach the executor
to lock only the tables of the latter kind when executing a cached
plan.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.comk
---
src/backend/optimizer/util/inherit.c | 6 ++++++
src/backend/parser/analyze.c | 7 +++----
src/include/nodes/parsenodes.h | 9 +++++++--
3 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..9bac07bf40 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -492,6 +492,12 @@ expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
}
else
childrte->inh = false;
+ /*
+ * Mark child tables as not being directly mentioned in the query. This
+ * allows the executor's ExecGetRangeTableRelation() to conveniently
+ * identify it as an inheritance child table.
+ */
+ childrte->inFromCl = false;
childrte->securityQuals = NIL;
/*
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4006632092..bcf6fcdde2 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -3267,10 +3267,9 @@ transformLockingClause(ParseState *pstate, Query *qry, LockingClause *lc,
/*
* Lock all regular tables used in query and its subqueries. We
* examine inFromCl to exclude auto-added RTEs, particularly NEW/OLD
- * in rules. This is a bit of an abuse of a mostly-obsolete flag, but
- * it's convenient. We can't rely on the namespace mechanism that has
- * largely replaced inFromCl, since for example we need to lock
- * base-relation RTEs even if they are masked by upper joins.
+ * in rules. We can't rely on the namespace mechanism since for
+ * example we need to lock base-relation RTEs even if they are masked
+ * by upper joins.
*/
i = 0;
foreach(rt, qry->rtable)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fe003ded50..72f2b0c04f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -994,11 +994,16 @@ typedef struct PartitionCmd
*
* inFromCl marks those range variables that are listed in the FROM clause.
* It's false for RTEs that are added to a query behind the scenes, such
- * as the NEW and OLD variables for a rule, or the subqueries of a UNION.
+ * as the NEW and OLD variables for a rule, or the subqueries of a UNION,
+ * or the RTEs of inheritance child tables that are added by the planner.
* This flag is not used during parsing (except in transformLockingClause,
* q.v.); the parser now uses a separate "namespace" data structure to
* control visibility. But it is needed by ruleutils.c to determine
- * whether RTEs should be shown in decompiled queries.
+ * whether RTEs should be shown in decompiled queries. It is used by the
+ * executor to determine that a given RTE_RELATION entry belongs to a table
+ * directly mentioned in the query or to a child table added by the planner.
+ * It needs to know that for the case where the child tables in a plan need
+ * to be locked.
*
* securityQuals is a list of security barrier quals (boolean expressions),
* to be tested in the listed order before returning a row from the
--
2.35.3
[application/octet-stream] v44-0006-Track-opened-range-table-relations-in-a-List-in-.patch (2.4K, 3-v44-0006-Track-opened-range-table-relations-in-a-List-in-.patch)
download | inline diff:
From 4f73533573cb5959b15268455605816c0316d0e6 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:49 +0900
Subject: [PATCH v44 6/6] Track opened range table relations in a List in
EState
This makes ExecCloseRangeTableRelations faster when there are many
relations in the range table but only a few are opened during
execution, such as when run-time pruning kicks in on an Append
containing thousands of partition subplans.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/executor/execMain.c | 9 +++++----
src/backend/executor/execUtils.c | 2 ++
src/include/nodes/execnodes.h | 2 ++
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index dcfbf58495..c574cd3cdc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1646,12 +1646,13 @@ ExecCloseResultRelations(EState *estate)
void
ExecCloseRangeTableRelations(EState *estate)
{
- int i;
+ ListCell *lc;
- for (i = 0; i < estate->es_range_table_size; i++)
+ foreach(lc, estate->es_opened_relations)
{
- if (estate->es_relations[i])
- table_close(estate->es_relations[i], NoLock);
+ Relation rel = lfirst(lc);
+
+ table_close(rel, NoLock);
}
}
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index ee12235b2f..5ae993e29c 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -839,6 +839,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
}
estate->es_relations[rti - 1] = rel;
+ estate->es_opened_relations = lappend(estate->es_opened_relations,
+ rel);
}
return rel;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 20c1bacae1..c519a6d5dc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -619,6 +619,8 @@ typedef struct EState
Index es_range_table_size; /* size of the range table arrays */
Relation *es_relations; /* Array of per-range-table-entry Relation
* pointers, or NULL if not yet opened */
+ List *es_opened_relations; /* List of non-NULL entries in
+ * es_relations in no specific order */
struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
--
2.35.3
[application/octet-stream] v44-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch (25.8K, 4-v44-0002-Refactoring-to-move-ExecutorStart-calls-to-be-ne.patch)
download | inline diff:
From 0d1067505ed1c49d4a75ad5d7f4eec4a19d7b5d6 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Thu, 3 Aug 2023 12:34:31 +0900
Subject: [PATCH v44 2/6] Refactoring to move ExecutorStart() calls to be near
GetCachedPlan()
An upcoming patch will make ExecutorStart() detect the invalidation
of a CachedPlan when initializing the plan tree contained in it. A
caller must retry with a new CachedPlan when ExecutorStart() detects
an invalidation. Having the ExecutorStart() in the same or nearby
as GetCachedPlan() makes it more convenient to implement the replan
loop.
The following sites have thus been modified:
* The ExecutorStart() call in ExplainOnePlan() is moved, along with
CreateQueryDesc(), into a new function ExplainQueryDesc(), which its
callers now call before calling it.
* The ExecutorStart() call in _SPI_pquery() is moved to its caller
_SPI_execute_plan().
* The ExecutorStart() call in PortalRunMulti() is moved to
PortalStart(). This requires a new List field in PortalData to
store the QueryDescs created in PortalStart() and the associated
memory context field. One unintended consequence is that the
CommandCounterIncrement() between queries in PORTAL_MULTI_QUERY
cases is now done in the loop in PortalStart() and not in
PortalRunMulti(). That still seems to work because the Snapshot
registered in QueryDesc/EState is updated to account for the
CCI().
---
src/backend/commands/explain.c | 121 ++++++-----
src/backend/commands/prepare.c | 12 +-
src/backend/executor/spi.c | 27 +--
src/backend/tcop/pquery.c | 311 +++++++++++++----------------
src/backend/utils/mmgr/portalmem.c | 9 +
src/include/commands/explain.h | 6 +-
src/include/utils/portal.h | 2 +
7 files changed, 250 insertions(+), 238 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8570b14f62..59d57f9c10 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
else
{
PlannedStmt *plan;
+ QueryDesc *queryDesc;
instr_time planstart,
planduration;
BufferUsage bufusage_start,
@@ -415,12 +416,77 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
+ queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ params, queryEnv);
+ Assert(queryDesc);
+
/* run it (if needed) and produce output */
- ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
+ ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv,
&planduration, (es->buffers ? &bufusage : NULL));
}
}
+/*
+ * ExplainQueryDesc
+ * Set up QueryDesc for EXPLAINing a given plan
+ */
+QueryDesc *
+ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv)
+{
+ QueryDesc *queryDesc;
+ DestReceiver *dest;
+ int eflags;
+ int instrument_option = 0;
+
+ /*
+ * Normally we discard the query's output, but if explaining CREATE TABLE
+ * AS, we'd better use the appropriate tuple receiver.
+ */
+ if (into)
+ dest = CreateIntoRelDestReceiver(into);
+ else
+ dest = None_Receiver;
+
+ if (es->analyze && es->timing)
+ instrument_option |= INSTRUMENT_TIMER;
+ else if (es->analyze)
+ instrument_option |= INSTRUMENT_ROWS;
+
+ if (es->buffers)
+ instrument_option |= INSTRUMENT_BUFFERS;
+ if (es->wal)
+ instrument_option |= INSTRUMENT_WAL;
+
+ /*
+ * Use a snapshot with an updated command ID to ensure this query sees
+ * results of any previously executed queries.
+ */
+ PushCopiedSnapshot(GetActiveSnapshot());
+ UpdateActiveSnapshotCommandId();
+
+ /* Create a QueryDesc for the query */
+ queryDesc = CreateQueryDesc(stmt, queryString,
+ GetActiveSnapshot(), InvalidSnapshot,
+ dest, params, queryEnv, instrument_option);
+
+ /* Select execution options */
+ if (es->analyze)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_EXPLAIN_ONLY;
+ if (es->generic)
+ eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
+ if (into)
+ eflags |= GetIntoRelEFlags(into);
+
+ /* Call ExecutorStart to prepare the plan for execution. */
+ ExecutorStart(queryDesc, eflags);
+
+ return queryDesc;
+}
+
/*
* ExplainOneUtility -
* print out the execution plan for one utility statement
@@ -524,29 +590,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
* to call it.
*/
void
-ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
+ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
const BufferUsage *bufusage)
{
- DestReceiver *dest;
- QueryDesc *queryDesc;
instr_time starttime;
double totaltime = 0;
- int eflags;
- int instrument_option = 0;
- Assert(plannedstmt->commandType != CMD_UTILITY);
-
- if (es->analyze && es->timing)
- instrument_option |= INSTRUMENT_TIMER;
- else if (es->analyze)
- instrument_option |= INSTRUMENT_ROWS;
-
- if (es->buffers)
- instrument_option |= INSTRUMENT_BUFFERS;
- if (es->wal)
- instrument_option |= INSTRUMENT_WAL;
+ Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY);
/*
* We always collect timing for the entire statement, even when node-level
@@ -555,40 +608,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
INSTR_TIME_SET_CURRENT(starttime);
- /*
- * Use a snapshot with an updated command ID to ensure this query sees
- * results of any previously executed queries.
- */
- PushCopiedSnapshot(GetActiveSnapshot());
- UpdateActiveSnapshotCommandId();
-
- /*
- * Normally we discard the query's output, but if explaining CREATE TABLE
- * AS, we'd better use the appropriate tuple receiver.
- */
- if (into)
- dest = CreateIntoRelDestReceiver(into);
- else
- dest = None_Receiver;
-
- /* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(plannedstmt, queryString,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, instrument_option);
-
- /* Select execution options */
- if (es->analyze)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_EXPLAIN_ONLY;
- if (es->generic)
- eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
- if (into)
- eflags |= GetIntoRelEFlags(into);
-
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, eflags);
-
/* Execute the plan for statistics if asked for */
if (es->analyze)
{
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 18f70319fc..1e9a98ad6e 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -639,8 +639,16 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
PlannedStmt *pstmt = lfirst_node(PlannedStmt, p);
if (pstmt->commandType != CMD_UTILITY)
- ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ {
+ QueryDesc *queryDesc;
+
+ queryDesc = ExplainQueryDesc(pstmt, queryString,
+ into, es, paramLI, queryEnv);
+ Assert(queryDesc != NULL);
+ ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
+ queryEnv, &planduration,
+ (es->buffers ? &bufusage : NULL));
+ }
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 33975687b3..d36ca35d3a 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes,
Datum *Values, const char *Nulls);
-static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount);
+static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount);
static void _SPI_error_callback(void *arg);
@@ -2661,6 +2661,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
{
QueryDesc *qdesc;
Snapshot snap;
+ int eflags;
if (ActiveSnapshotSet())
snap = GetActiveSnapshot();
@@ -2674,8 +2675,17 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
options->params,
_SPI_current->queryEnv,
0);
- res = _SPI_pquery(qdesc, fire_triggers,
- canSetTag ? options->tcount : 0);
+
+ /* Select execution options */
+ if (fire_triggers)
+ eflags = 0; /* default run-to-completion flags */
+ else
+ eflags = EXEC_FLAG_SKIP_TRIGGERS;
+
+ ExecutorStart(qdesc, eflags);
+
+ res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
+
FreeQueryDesc(qdesc);
}
else
@@ -2850,10 +2860,9 @@ _SPI_convert_params(int nargs, Oid *argtypes,
}
static int
-_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
+_SPI_pquery(QueryDesc *queryDesc, uint64 tcount)
{
int operation = queryDesc->operation;
- int eflags;
int res;
switch (operation)
@@ -2897,14 +2906,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount)
ResetUsage();
#endif
- /* Select execution options */
- if (fire_triggers)
- eflags = 0; /* default run-to-completion flags */
- else
- eflags = EXEC_FLAG_SKIP_TRIGGERS;
-
- ExecutorStart(queryDesc, eflags);
-
ExecutorRun(queryDesc, ForwardScanDirection, tcount, true);
_SPI_current->processed = queryDesc->estate->es_processed;
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 5565f200c3..701808f303 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -19,6 +19,7 @@
#include "access/xact.h"
#include "commands/prepare.h"
+#include "executor/execdesc.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
#include "pg_trace.h"
@@ -35,12 +36,6 @@
Portal ActivePortal = NULL;
-static void ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc);
static void FillPortalStore(Portal portal, bool isTopLevel);
static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count,
DestReceiver *dest);
@@ -116,86 +111,6 @@ FreeQueryDesc(QueryDesc *qdesc)
}
-/*
- * ProcessQuery
- * Execute a single plannable query within a PORTAL_MULTI_QUERY,
- * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal
- *
- * plan: the plan tree for the query
- * sourceText: the source text of the query
- * params: any parameters needed
- * dest: where to send results
- * qc: where to store the command completion status data.
- *
- * qc may be NULL if caller doesn't want a status string.
- *
- * Must be called in a memory context that will be reset or deleted on
- * error; otherwise the executor's memory usage will be leaked.
- */
-static void
-ProcessQuery(PlannedStmt *plan,
- const char *sourceText,
- ParamListInfo params,
- QueryEnvironment *queryEnv,
- DestReceiver *dest,
- QueryCompletion *qc)
-{
- QueryDesc *queryDesc;
-
- /*
- * Create the QueryDesc object
- */
- queryDesc = CreateQueryDesc(plan, sourceText,
- GetActiveSnapshot(), InvalidSnapshot,
- dest, params, queryEnv, 0);
-
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
- ExecutorStart(queryDesc, 0);
-
- /*
- * Run the plan to completion.
- */
- ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
-
- /*
- * Build command completion status data, if caller wants one.
- */
- if (qc)
- {
- switch (queryDesc->operation)
- {
- case CMD_SELECT:
- SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed);
- break;
- case CMD_INSERT:
- SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed);
- break;
- case CMD_UPDATE:
- SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed);
- break;
- case CMD_DELETE:
- SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed);
- break;
- case CMD_MERGE:
- SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed);
- break;
- default:
- SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed);
- break;
- }
- }
-
- /*
- * Now, we close down all the scans and free allocated resources.
- */
- ExecutorFinish(queryDesc);
- ExecutorEnd(queryDesc);
-
- FreeQueryDesc(queryDesc);
-}
-
/*
* ChoosePortalStrategy
* Select portal execution strategy given the intended statement list.
@@ -435,10 +350,9 @@ PortalStart(Portal portal, ParamListInfo params,
{
Portal saveActivePortal;
ResourceOwner saveResourceOwner;
- MemoryContext savePortalContext;
MemoryContext oldContext;
QueryDesc *queryDesc;
- int myeflags;
+ int myeflags = 0;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -448,15 +362,13 @@ PortalStart(Portal portal, ParamListInfo params,
*/
saveActivePortal = ActivePortal;
saveResourceOwner = CurrentResourceOwner;
- savePortalContext = PortalContext;
PG_TRY();
{
ActivePortal = portal;
if (portal->resowner)
CurrentResourceOwner = portal->resowner;
- PortalContext = portal->portalContext;
- oldContext = MemoryContextSwitchTo(PortalContext);
+ oldContext = MemoryContextSwitchTo(portal->queryContext);
/* Must remember portal param list, if any */
portal->portalParams = params;
@@ -472,6 +384,8 @@ PortalStart(Portal portal, ParamListInfo params,
switch (portal->strategy)
{
case PORTAL_ONE_SELECT:
+ case PORTAL_ONE_RETURNING:
+ case PORTAL_ONE_MOD_WITH:
/* Must set snapshot before starting executor. */
if (snapshot)
@@ -489,8 +403,8 @@ PortalStart(Portal portal, ParamListInfo params,
*/
/*
- * Create QueryDesc in portal's context; for the moment, set
- * the destination to DestNone.
+ * Create QueryDesc in portal->queryContext; for the moment,
+ * set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
portal->sourceText,
@@ -501,30 +415,41 @@ PortalStart(Portal portal, ParamListInfo params,
portal->queryEnv,
0);
+ /* Remember for PortalRunMulti(). */
+ if (portal->strategy == PORTAL_ONE_RETURNING ||
+ portal->strategy == PORTAL_ONE_MOD_WITH)
+ portal->qdescs = list_make1(queryDesc);
+
/*
* If it's a scrollable cursor, executor needs to support
* REWIND and backwards scan, as well as whatever the caller
* might've asked for.
*/
- if (portal->cursorOptions & CURSOR_OPT_SCROLL)
+ if (portal->strategy == PORTAL_ONE_SELECT &&
+ (portal->cursorOptions & CURSOR_OPT_SCROLL))
myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD;
else
myeflags = eflags;
- /*
- * Call ExecutorStart to prepare the plan for execution
- */
+ /* Call ExecutorStart to prepare the plan for execution. */
ExecutorStart(queryDesc, myeflags);
/*
- * This tells PortalCleanup to shut down the executor
+ * This tells PortalCleanup to shut down the executor, though
+ * not needed for queries handled by PortalRunMulti().
*/
- portal->queryDesc = queryDesc;
+ if (portal->strategy == PORTAL_ONE_SELECT)
+ portal->queryDesc = queryDesc;
/*
- * Remember tuple descriptor (computed by ExecutorStart)
+ * Remember tuple descriptor (computed by ExecutorStart),
+ * though make it independent of QueryDesc for queries handled
+ * by PortalRunMulti().
*/
- portal->tupDesc = queryDesc->tupDesc;
+ if (portal->strategy != PORTAL_ONE_SELECT)
+ portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc);
+ else
+ portal->tupDesc = queryDesc->tupDesc;
/*
* Reset cursor position data to "start of query"
@@ -536,29 +461,6 @@ PortalStart(Portal portal, ParamListInfo params,
PopActiveSnapshot();
break;
- case PORTAL_ONE_RETURNING:
- case PORTAL_ONE_MOD_WITH:
-
- /*
- * We don't start the executor until we are told to run the
- * portal. We do need to set up the result tupdesc.
- */
- {
- PlannedStmt *pstmt;
-
- pstmt = PortalGetPrimaryStmt(portal);
- portal->tupDesc =
- ExecCleanTypeFromTL(pstmt->planTree->targetlist);
- }
-
- /*
- * Reset cursor position data to "start of query"
- */
- portal->atStart = true;
- portal->atEnd = false; /* allow fetches */
- portal->portalPos = 0;
- break;
-
case PORTAL_UTIL_SELECT:
/*
@@ -581,7 +483,69 @@ PortalStart(Portal portal, ParamListInfo params,
break;
case PORTAL_MULTI_QUERY:
- /* Need do nothing now */
+ {
+ ListCell *lc;
+ bool first = true;
+
+ myeflags = eflags;
+ foreach(lc, portal->stmts)
+ {
+ PlannedStmt *plan = lfirst_node(PlannedStmt, lc);
+ bool is_utility = (plan->utilityStmt != NULL);
+
+ /*
+ * Push the snapshot to be used by the executor.
+ */
+ if (!is_utility)
+ {
+ /*
+ * Must copy the snapshot for all statements
+ * except thec first as we'll need to update its
+ * command ID.
+ */
+ if (!first)
+ PushCopiedSnapshot(GetTransactionSnapshot());
+ else
+ PushActiveSnapshot(GetTransactionSnapshot());
+ }
+
+ /*
+ * From the 2nd statement onwards, update the command
+ * ID and the snapshot to match.
+ */
+ if (!first)
+ {
+ CommandCounterIncrement();
+ UpdateActiveSnapshotCommandId();
+ }
+
+ first = false;
+
+ /*
+ * Create the QueryDesc. DestReceiver will be set in
+ * PortalRunMulti() before calling ExecutorRun().
+ */
+ queryDesc = CreateQueryDesc(plan,
+ portal->sourceText,
+ !is_utility ?
+ GetActiveSnapshot() :
+ InvalidSnapshot,
+ InvalidSnapshot,
+ NULL,
+ params,
+ portal->queryEnv, 0);
+
+ /* Remember for PortalRunMulti() */
+ portal->qdescs = lappend(portal->qdescs, queryDesc);
+
+ if (is_utility)
+ continue;
+
+ ExecutorStart(queryDesc, myeflags);
+ PopActiveSnapshot();
+ }
+ }
+
portal->tupDesc = NULL;
break;
}
@@ -594,7 +558,6 @@ PortalStart(Portal portal, ParamListInfo params,
/* Restore global vars and propagate error */
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
PG_RE_THROW();
}
@@ -604,7 +567,6 @@ PortalStart(Portal portal, ParamListInfo params,
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- PortalContext = savePortalContext;
portal->status = PORTAL_READY;
}
@@ -1193,7 +1155,7 @@ PortalRunMulti(Portal portal,
QueryCompletion *qc)
{
bool active_snapshot_set = false;
- ListCell *stmtlist_item;
+ ListCell *qdesc_item;
/*
* If the destination is DestRemoteExecute, change to DestNone. The
@@ -1214,9 +1176,10 @@ PortalRunMulti(Portal portal,
* Loop to handle the individual queries generated from a single parsetree
* by analysis and rewrite.
*/
- foreach(stmtlist_item, portal->stmts)
+ foreach(qdesc_item, portal->qdescs)
{
- PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item);
+ QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item);
+ PlannedStmt *pstmt = qdesc->plannedstmt;
/*
* If we got a cancel signal in prior command, quit
@@ -1233,33 +1196,26 @@ PortalRunMulti(Portal portal,
if (log_executor_stats)
ResetUsage();
- /*
- * Must always have a snapshot for plannable queries. First time
- * through, take a new snapshot; for subsequent queries in the
- * same portal, just update the snapshot's copy of the command
- * counter.
- */
+ /* Push the snapshot for plannable queries. */
if (!active_snapshot_set)
{
- Snapshot snapshot = GetTransactionSnapshot();
+ Snapshot snapshot = qdesc->snapshot;
- /* If told to, register the snapshot and save in portal */
+ /*
+ * If told to, register the snapshot and save in portal
+ *
+ * Note that the command ID of qdesc->snapshot for 2nd query
+ * onwards would have been updated in PortalStart() to account
+ * for CCI() done between queries, but it's OK that here we
+ * don't likewise update holdSnapshot's command ID.
+ */
if (setHoldSnapshot)
{
snapshot = RegisterSnapshot(snapshot);
portal->holdSnapshot = snapshot;
}
- /*
- * We can't have the holdSnapshot also be the active one,
- * because UpdateActiveSnapshotCommandId would complain. So
- * force an extra snapshot copy. Plain PushActiveSnapshot
- * would have copied the transaction snapshot anyway, so this
- * only adds a copy step when setHoldSnapshot is true. (It's
- * okay for the command ID of the active snapshot to diverge
- * from what holdSnapshot has.)
- */
- PushCopiedSnapshot(snapshot);
+ PushActiveSnapshot(snapshot);
/*
* As for PORTAL_ONE_SELECT portals, it does not seem
@@ -1268,26 +1224,39 @@ PortalRunMulti(Portal portal,
active_snapshot_set = true;
}
- else
- UpdateActiveSnapshotCommandId();
+ /*
+ * Run the plan to completion.
+ */
+ qdesc->dest = dest;
+ ExecutorRun(qdesc, ForwardScanDirection, 0, true);
+
+ /*
+ * Build command completion status data if needed.
+ */
if (pstmt->canSetTag)
{
- /* statement can set tag string */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- dest, qc);
- }
- else
- {
- /* stmt added by rewrite cannot set tag */
- ProcessQuery(pstmt,
- portal->sourceText,
- portal->portalParams,
- portal->queryEnv,
- altdest, NULL);
+ switch (qdesc->operation)
+ {
+ case CMD_SELECT:
+ SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed);
+ break;
+ case CMD_INSERT:
+ SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed);
+ break;
+ case CMD_UPDATE:
+ SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed);
+ break;
+ case CMD_DELETE:
+ SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed);
+ break;
+ case CMD_MERGE:
+ SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed);
+ break;
+ default:
+ SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed);
+ break;
+ }
}
if (log_executor_stats)
@@ -1342,12 +1311,12 @@ PortalRunMulti(Portal portal,
if (portal->stmts == NIL)
break;
- /*
- * Increment command counter between queries, but not after the last
- * one.
- */
- if (lnext(portal->stmts, stmtlist_item) != NULL)
- CommandCounterIncrement();
+ if (qdesc->estate)
+ {
+ ExecutorFinish(qdesc);
+ ExecutorEnd(qdesc);
+ }
+ FreeQueryDesc(qdesc);
}
/* Pop the snapshot if we pushed one. */
diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c
index 06dfa85f04..0cad450dcd 100644
--- a/src/backend/utils/mmgr/portalmem.c
+++ b/src/backend/utils/mmgr/portalmem.c
@@ -201,6 +201,13 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
portal->portalContext = AllocSetContextCreate(TopPortalContext,
"PortalContext",
ALLOCSET_SMALL_SIZES);
+ /*
+ * initialize portal's query context to store QueryDescs created during
+ * PortalStart() and then used in PortalRun().
+ */
+ portal->queryContext = AllocSetContextCreate(TopPortalContext,
+ "PortalQueryContext",
+ ALLOCSET_SMALL_SIZES);
/* create a resource owner for the portal */
portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner,
@@ -224,6 +231,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent)
/* for named portals reuse portal->name copy */
MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>");
+ MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>");
return portal;
}
@@ -594,6 +602,7 @@ PortalDrop(Portal portal, bool isTopCommit)
/* release subsidiary storage */
MemoryContextDelete(portal->portalContext);
+ MemoryContextDelete(portal->queryContext);
/* release portal struct (it's in TopPortalContext) */
pfree(portal);
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 3d3e632a0c..08ea852b65 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+ const char *queryString, IntoClause *into, ExplainState *es,
+ ParamListInfo params, QueryEnvironment *queryEnv);
+extern void ExplainOnePlan(QueryDesc *queryDesc,
+ IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h
index aa08b1e0fc..af059e30f8 100644
--- a/src/include/utils/portal.h
+++ b/src/include/utils/portal.h
@@ -138,6 +138,8 @@ typedef struct PortalData
QueryCompletion qc; /* command completion data for executed query */
List *stmts; /* list of PlannedStmts */
CachedPlan *cplan; /* CachedPlan, if stmts are from one */
+ List *qdescs; /* list of QueryDescs */
+ MemoryContext queryContext; /* memory for QueryDescs and children */
ParamListInfo portalParams; /* params to pass to query */
QueryEnvironment *queryEnv; /* environment for query */
--
2.35.3
[application/octet-stream] v44-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch (21.2K, 5-v44-0003-Add-field-to-store-parent-relids-to-Append-Merge.patch)
download | inline diff:
From 10c7bbe9f1a489d0fcfeaf027a7df919fed490c8 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:31 +0900
Subject: [PATCH v44 3/6] Add field to store parent relids to
Append/MergeAppend
There's no way currently in the executor to tell if the child
subplans of Append/MergeAppend are scanning partitions, and if
they indeed do, what the RT indexes of their parent/ancestor tables
are. Executor doesn't need to see their RT indexes except for
run-time pruning, in which case they can can be found in the
PartitionPruneInfo, but a future commit will create a need for
them to be available at all times for the purpose of locking
those parent/ancestor tables when executing a cached plan.
The code to look up partitioned parent relids for a given list of
partition scan subpaths of an Append/MergeAppend is already present
in make_partition_pruneinfo() but it's local to partprune.c. This
commit refactors that code into its own function called
add_append_subpath_partrelids() defined in appendinfo.c and
generalizes it to consider child join and aggregate paths. To
facilitate looking up of parent rels of child grouping rels in
add_append_subpath_partrelids(), parent links are now also set in
the RelOptInfos of child grouping rels too, like they are in
those of child base and join rels.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
src/backend/optimizer/plan/createplan.c | 41 ++++++--
src/backend/optimizer/plan/planner.c | 3 +
src/backend/optimizer/plan/setrefs.c | 4 +
src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++
src/backend/partitioning/partprune.c | 124 +++-------------------
src/include/nodes/plannodes.h | 14 +++
src/include/optimizer/appendinfo.h | 3 +
src/include/partitioning/partprune.h | 3 +-
8 files changed, 203 insertions(+), 123 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..8ac1d3909b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
@@ -1210,6 +1211,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
Oid *nodeCollations = NULL;
bool *nodeNullsFirst = NULL;
bool consider_async = false;
+ List *allpartrelids = NIL;
/*
* The subpaths list could be empty, if every child was proven empty by
@@ -1351,15 +1353,23 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
++nasyncplans;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ plan->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1380,7 +1390,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
partpruneinfo =
make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
plan->appendplans = subplans;
@@ -1426,6 +1437,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
ListCell *subpaths;
RelOptInfo *rel = best_path->path.parent;
PartitionPruneInfo *partpruneinfo = NULL;
+ List *allpartrelids = NIL;
/*
* We don't have the actual creation of the MergeAppend node split out
@@ -1515,15 +1527,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
subplan = (Plan *) sort;
}
+ /*
+ * Find partitioned parent rel(s) of the subpath's rel(s).
+ */
+ allpartrelids = add_append_subpath_partrelids(root, subpath, rel,
+ allpartrelids);
+
subplans = lappend(subplans, subplan);
}
+ node->allpartrelids = allpartrelids;
+
/*
- * If any quals exist, they may be useful to perform further partition
- * pruning during execution. Gather information needed by the executor to
- * do partition pruning.
+ * If scanning partitions, check if there are quals that may be useful to
+ * perform further partition pruning during execution. Gather information
+ * needed by the executor to do partition pruning.
*/
- if (enable_partition_pruning)
+ if (enable_partition_pruning && allpartrelids != NIL)
{
List *prunequal;
@@ -1535,7 +1555,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
- prunequal);
+ prunequal,
+ allpartrelids);
}
node->mergeplans = subplans;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..f97bc09113 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7855,8 +7855,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ /* Mark as child of grouped_rel. */
+ child_grouped_rel->parent = grouped_rel;
if (child_partially_grouped_rel)
{
+ child_partially_grouped_rel->parent = grouped_rel;
partially_grouped_live_children =
lappend(partially_grouped_live_children,
child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..854dd7c8af 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1766,6 +1766,8 @@ set_append_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) aplan, rtoffset);
aplan->apprelids = offset_relid_set(aplan->apprelids, rtoffset);
+ foreach(l, aplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (aplan->part_prune_info)
{
@@ -1842,6 +1844,8 @@ set_mergeappend_references(PlannerInfo *root,
set_dummy_tlist_references((Plan *) mplan, rtoffset);
mplan->apprelids = offset_relid_set(mplan->apprelids, rtoffset);
+ foreach(l, mplan->allpartrelids)
+ lfirst(l) = offset_relid_set((Relids) lfirst(l), rtoffset);
if (mplan->part_prune_info)
{
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f456b3b0a4..5bd8e82b9b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -41,6 +41,7 @@ static void make_inh_translation_list(Relation oldrelation,
AppendRelInfo *appinfo);
static Node *adjust_appendrel_attrs_mutator(Node *node,
adjust_appendrel_attrs_context *context);
+static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
/*
@@ -1035,3 +1036,136 @@ distribute_row_identity_vars(PlannerInfo *root)
}
}
}
+
+/*
+ * add_append_subpath_partrelids
+ * Look up a child subpath's rel's partitioned parent relids up to
+ * parentrel and add the bitmapset containing those into
+ * 'allpartrelids'
+ */
+List *
+add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids)
+{
+ RelOptInfo *prel = subpath->parent;
+ Relids partrelids = NULL;
+
+ /* Nothing to do if there's no parent to begin with. */
+ if (!IS_OTHER_REL(prel))
+ return allpartrelids;
+
+ /*
+ * Traverse up to the pathrel's topmost partitioned parent, collecting
+ * parent relids as we go; but stop if we reach parentrel. (Normally, a
+ * pathrel's topmost partitioned parent is either parentrel or a UNION ALL
+ * appendrel child of parentrel. But when handling partitionwise joins of
+ * multi-level partitioning trees, we can see an append path whose
+ * parentrel is an intermediate partitioned table.)
+ */
+ do
+ {
+ Relids parent_relids = NULL;
+
+ /*
+ * For simple child rels, we can simply set the parent_relids to
+ * prel->parent->relids. But for partitionwise join and aggregate
+ * child rels, while we can use prel->parent to move up the tree,
+ * parent_relids must be found the hard way through AppendInfoInfos,
+ * because 1) a joinrel's relids may point to RTE_JOIN entries,
+ * 2) topmost parent grouping rel's relids field is NULL.
+ */
+ if (IS_SIMPLE_REL(prel))
+ {
+ prel = prel->parent;
+ /* Stop once we reach the root partitioned rel. */
+ if (!IS_PARTITIONED_REL(prel))
+ break;
+ parent_relids = bms_add_members(parent_relids, prel->relids);
+ }
+ else
+ {
+ AppendRelInfo **appinfos;
+ int nappinfos,
+ i;
+
+ appinfos = find_appinfos_by_relids(root, prel->relids,
+ &nappinfos);
+ for (i = 0; i < nappinfos; i++)
+ {
+ AppendRelInfo *appinfo = appinfos[i];
+
+ parent_relids = bms_add_member(parent_relids,
+ appinfo->parent_relid);
+ }
+ pfree(appinfos);
+ prel = prel->parent;
+ }
+ /* accept this level as an interesting parent */
+ partrelids = bms_add_members(partrelids, parent_relids);
+ if (prel == parentrel)
+ break; /* don't traverse above parentrel */
+ } while (IS_OTHER_REL(prel));
+
+ if (partrelids == NULL)
+ return allpartrelids;
+
+ return add_part_relids(allpartrelids, partrelids);
+}
+
+/*
+ * add_part_relids
+ * Add new info to a list of Bitmapsets of partitioned relids.
+ *
+ * Within 'allpartrelids', there is one Bitmapset for each topmost parent
+ * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
+ * parent as well as its relevant non-leaf child partitions. Since (by
+ * construction of the rangetable list) parent partitions must have lower
+ * RT indexes than their children, we can distinguish the topmost parent
+ * as being the lowest set bit in the Bitmapset.
+ *
+ * 'partrelids' contains the RT indexes of a parent partitioned rel, and
+ * possibly some non-leaf children, that are newly identified as parents of
+ * some subpath rel passed to make_partition_pruneinfo(). These are added
+ * to an appropriate member of 'allpartrelids'.
+ *
+ * Note that the list contains only RT indexes of partitioned tables that
+ * are parents of some scan-level relation appearing in the 'subpaths' that
+ * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
+ * not allowed to be higher than the 'parentrel' associated with the append
+ * path. In this way, we avoid expending cycles on partitioned rels that
+ * can't contribute useful pruning information for the problem at hand.
+ * (It is possible for 'parentrel' to be a child partitioned table, and it
+ * is also possible for scan-level relations to be child partitioned tables
+ * rather than leaf partitions. Hence we must construct this relation set
+ * with reference to the particular append path we're dealing with, rather
+ * than looking at the full partitioning structure represented in the
+ * RelOptInfos.)
+ */
+static List *
+add_part_relids(List *allpartrelids, Bitmapset *partrelids)
+{
+ Index targetpart;
+ ListCell *lc;
+
+ /* We can easily get the lowest set bit this way: */
+ targetpart = bms_next_member(partrelids, -1);
+ Assert(targetpart > 0);
+
+ /* Look for a matching topmost parent */
+ foreach(lc, allpartrelids)
+ {
+ Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
+ Index currtarget = bms_next_member(currpartrelids, -1);
+
+ if (targetpart == currtarget)
+ {
+ /* Found a match, so add any new RT indexes to this hierarchy */
+ currpartrelids = bms_add_members(currpartrelids, partrelids);
+ lfirst(lc) = currpartrelids;
+ return allpartrelids;
+ }
+ }
+ /* No match, so add the new partition hierarchy to the list */
+ return lappend(allpartrelids, partrelids);
+}
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..213512a5f4 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -138,7 +138,6 @@ typedef struct PruneStepResult
} PruneStepResult;
-static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids);
static List *make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
List *prunequal,
@@ -218,33 +217,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* of scan paths for its child rels.
* 'prunequal' is a list of potential pruning quals (i.e., restriction
* clauses that are applicable to the appendrel).
+ * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents
+ * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every
+ * partition tree involved.
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths,
- List *prunequal)
+ List *prunequal,
+ List *allpartrelids)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
- List *allpartrelids;
List *prunerelinfos;
int *relid_subplan_map;
ListCell *lc;
int i;
+ Assert(list_length(allpartrelids) > 0);
+
/*
- * Scan the subpaths to see which ones are scans of partition child
- * relations, and identify their parent partitioned rels. (Note: we must
- * restrict the parent partitioned rels to be parentrel or children of
- * parentrel, otherwise we couldn't translate prunequal to match.)
- *
- * Also construct a temporary array to map from partition-child-relation
- * relid to the index in 'subpaths' of the scan plan for that partition.
+ * Construct a temporary array to map from partition-child-relation relid
+ * to the index in 'subpaths' of the scan plan for that partition.
* (Use of "subplan" rather than "subpath" is a bit of a misnomer, but
* we'll let it stand.) For convenience, we use 1-based indexes here, so
* that zero can represent an un-filled array entry.
*/
- allpartrelids = NIL;
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
i = 1;
@@ -253,50 +251,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
Path *path = (Path *) lfirst(lc);
RelOptInfo *pathrel = path->parent;
- /* We don't consider partitioned joins here */
- if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- {
- RelOptInfo *prel = pathrel;
- Bitmapset *partrelids = NULL;
-
- /*
- * Traverse up to the pathrel's topmost partitioned parent,
- * collecting parent relids as we go; but stop if we reach
- * parentrel. (Normally, a pathrel's topmost partitioned parent
- * is either parentrel or a UNION ALL appendrel child of
- * parentrel. But when handling partitionwise joins of
- * multi-level partitioning trees, we can see an append path whose
- * parentrel is an intermediate partitioned table.)
- */
- do
- {
- AppendRelInfo *appinfo;
-
- Assert(prel->relid < root->simple_rel_array_size);
- appinfo = root->append_rel_array[prel->relid];
- prel = find_base_rel(root, appinfo->parent_relid);
- if (!IS_PARTITIONED_REL(prel))
- break; /* reached a non-partitioned parent */
- /* accept this level as an interesting parent */
- partrelids = bms_add_member(partrelids, prel->relid);
- if (prel == parentrel)
- break; /* don't traverse above parentrel */
- } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL);
-
- if (partrelids)
- {
- /*
- * Found some relevant parent partitions, which may or may not
- * overlap with partition trees we already found. Add new
- * information to the allpartrelids list.
- */
- allpartrelids = add_part_relids(allpartrelids, partrelids);
- /* Also record the subplan in relid_subplan_map[] */
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i;
- }
- }
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+ relid_subplan_map[pathrel->relid] = i;
i++;
}
@@ -362,63 +319,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
return pruneinfo;
}
-/*
- * add_part_relids
- * Add new info to a list of Bitmapsets of partitioned relids.
- *
- * Within 'allpartrelids', there is one Bitmapset for each topmost parent
- * partitioned rel. Each Bitmapset contains the RT indexes of the topmost
- * parent as well as its relevant non-leaf child partitions. Since (by
- * construction of the rangetable list) parent partitions must have lower
- * RT indexes than their children, we can distinguish the topmost parent
- * as being the lowest set bit in the Bitmapset.
- *
- * 'partrelids' contains the RT indexes of a parent partitioned rel, and
- * possibly some non-leaf children, that are newly identified as parents of
- * some subpath rel passed to make_partition_pruneinfo(). These are added
- * to an appropriate member of 'allpartrelids'.
- *
- * Note that the list contains only RT indexes of partitioned tables that
- * are parents of some scan-level relation appearing in the 'subpaths' that
- * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are
- * not allowed to be higher than the 'parentrel' associated with the append
- * path. In this way, we avoid expending cycles on partitioned rels that
- * can't contribute useful pruning information for the problem at hand.
- * (It is possible for 'parentrel' to be a child partitioned table, and it
- * is also possible for scan-level relations to be child partitioned tables
- * rather than leaf partitions. Hence we must construct this relation set
- * with reference to the particular append path we're dealing with, rather
- * than looking at the full partitioning structure represented in the
- * RelOptInfos.)
- */
-static List *
-add_part_relids(List *allpartrelids, Bitmapset *partrelids)
-{
- Index targetpart;
- ListCell *lc;
-
- /* We can easily get the lowest set bit this way: */
- targetpart = bms_next_member(partrelids, -1);
- Assert(targetpart > 0);
-
- /* Look for a matching topmost parent */
- foreach(lc, allpartrelids)
- {
- Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc);
- Index currtarget = bms_next_member(currpartrelids, -1);
-
- if (targetpart == currtarget)
- {
- /* Found a match, so add any new RT indexes to this hierarchy */
- currpartrelids = bms_add_members(currpartrelids, partrelids);
- lfirst(lc) = currpartrelids;
- return allpartrelids;
- }
- }
- /* No match, so add the new partition hierarchy to the list */
- return lappend(allpartrelids, partrelids);
-}
-
/*
* make_partitionedrel_pruneinfo
* Build a List of PartitionedRelPruneInfos, one for each interesting
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..7a5f3ba625 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -267,6 +267,13 @@ typedef struct Append
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * appendplans. The list contains a bitmapset for every partition tree
+ * covered by this Append.
+ */
+ List *allpartrelids;
+
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
@@ -291,6 +298,13 @@ typedef struct MergeAppend
List *mergeplans;
+ /*
+ * RTIs of all partitioned tables whose children are scanned by
+ * mergeplans. The list contains a bitmapset for every partition tree
+ * covered by this MergeAppend.
+ */
+ List *allpartrelids;
+
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index a05f91f77d..1621a7319a 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex,
RangeTblEntry *target_rte,
Relation target_relation);
extern void distribute_row_identity_vars(PlannerInfo *root);
+extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath,
+ RelOptInfo *parentrel,
+ List *allpartrelids);
#endif /* APPENDINFO_H */
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 8636e04e37..caa774a111 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -73,7 +73,8 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
- List *prunequal);
+ List *prunequal,
+ List *allpartrelids);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
List *pruning_steps);
--
2.35.3
[application/octet-stream] v44-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch (101.6K, 6-v44-0005-Delay-locking-of-child-tables-in-cached-plans-un.patch)
download | inline diff:
From a51b753af1c38e1a7750e9f09738d304ebd7de07 Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 4 Jul 2023 22:36:45 +0900
Subject: [PATCH v44 5/6] Delay locking of child tables in cached plans until
ExecutorStart()
Currently, GetCachedPlan() takes a lock on all relations contained in
a cached plan before returning it as a valid plan to its callers for
execution. One disadvantage is that if the plan contains partitions
that are prunable with conditions involving EXTERN parameters and
other stable expressions (known as "initial pruning"), many of them
would be locked unnecessarily, because only those that survive
initial pruning need to have been locked. Locking all partitions this
way causes significant delay when there are many partitions. Note
that initial pruning occurs during executor's initialization of the
plan, that is, ExecInitNode().
This commit rearranges things to move the locking of child tables
referenced in a cached plan to occur during ExecInitNode() so that
initial pruning in the ExecInitNode() subroutines of the plan nodes
that support pruning can eliminate any child tables that need not be
scanned and thus locked.
To determine that a given table is a child table,
ExecGetRangeTableRelation() now looks at the RTE's inFromCl field,
which is only true for tables that are directly mentioned in the
query but false for child tables. Note that any tables whose RTEs'
inFromCl is true would already have been locked by GetCachedPlan(),
so need not be locked again during execution.
If the locking of child tables causes the CachedPlan to go stale, that
is, its is_valid set to false by PlanCacheRelCallback() when an
invalidation message matching some child table contained in the plan
is processed, ExecInitNode() abandons the initialization of the
remaining nodes in the plan tree. In that case, InitPlan() returns
after setting QueryDesc.planstate to NULL to indicate to the caller
that no execution is possible with the plan tree as is. Also,
ExecutorStart() now returns true or false to indicate whether or not
QueryDesc.planstate points to a successfully initialized PlanState
tree. Call sites that use GetCachedPlan() to get the plan trees to
pass to the executor should now be prepared to retry in the cases
where ExecutorStart() returns false.
Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
---
contrib/auto_explain/auto_explain.c | 12 +-
.../pg_stat_statements/pg_stat_statements.c | 12 +-
contrib/postgres_fdw/postgres_fdw.c | 4 +
src/backend/commands/copyto.c | 7 +-
src/backend/commands/createas.c | 10 +-
src/backend/commands/explain.c | 33 +++-
src/backend/commands/extension.c | 4 +-
src/backend/commands/matview.c | 10 +-
src/backend/commands/portalcmds.c | 5 +-
src/backend/commands/prepare.c | 23 ++-
src/backend/executor/README | 37 +++++
src/backend/executor/execMain.c | 89 ++++++++--
src/backend/executor/execParallel.c | 14 +-
src/backend/executor/execPartition.c | 14 ++
src/backend/executor/execProcnode.c | 20 ++-
src/backend/executor/execUtils.c | 63 +++++--
src/backend/executor/functions.c | 5 +-
src/backend/executor/nodeAgg.c | 2 +
src/backend/executor/nodeAppend.c | 23 +++
src/backend/executor/nodeBitmapAnd.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 4 +
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeBitmapOr.c | 5 +-
src/backend/executor/nodeCustom.c | 2 +
src/backend/executor/nodeForeignscan.c | 4 +
src/backend/executor/nodeGather.c | 3 +
src/backend/executor/nodeGatherMerge.c | 2 +
src/backend/executor/nodeGroup.c | 2 +
src/backend/executor/nodeHash.c | 2 +
src/backend/executor/nodeHashjoin.c | 4 +
src/backend/executor/nodeIncrementalSort.c | 2 +
src/backend/executor/nodeIndexonlyscan.c | 11 +-
src/backend/executor/nodeIndexscan.c | 11 +-
src/backend/executor/nodeLimit.c | 2 +
src/backend/executor/nodeLockRows.c | 2 +
src/backend/executor/nodeMaterial.c | 2 +
src/backend/executor/nodeMemoize.c | 2 +
src/backend/executor/nodeMergeAppend.c | 23 +++
src/backend/executor/nodeMergejoin.c | 4 +
src/backend/executor/nodeModifyTable.c | 7 +
src/backend/executor/nodeNestloop.c | 4 +
src/backend/executor/nodeProjectSet.c | 2 +
src/backend/executor/nodeRecursiveunion.c | 4 +
src/backend/executor/nodeResult.c | 2 +
src/backend/executor/nodeSamplescan.c | 2 +
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/executor/nodeSetOp.c | 2 +
src/backend/executor/nodeSort.c | 2 +
src/backend/executor/nodeSubqueryscan.c | 2 +
src/backend/executor/nodeTidrangescan.c | 2 +
src/backend/executor/nodeTidscan.c | 2 +
src/backend/executor/nodeUnique.c | 2 +
src/backend/executor/nodeWindowAgg.c | 2 +
src/backend/executor/spi.c | 26 ++-
src/backend/storage/lmgr/lmgr.c | 45 +++++
src/backend/tcop/postgres.c | 18 +-
src/backend/tcop/pquery.c | 49 +++++-
src/backend/utils/cache/lsyscache.c | 21 +++
src/backend/utils/cache/plancache.c | 156 +++++++-----------
src/include/commands/explain.h | 3 +-
src/include/executor/execdesc.h | 4 +
src/include/executor/executor.h | 19 ++-
src/include/nodes/execnodes.h | 2 +
src/include/storage/lmgr.h | 1 +
src/include/tcop/pquery.h | 2 +-
src/include/utils/lsyscache.h | 1 +
src/include/utils/plancache.h | 14 ++
src/test/modules/delay_execution/Makefile | 3 +-
.../modules/delay_execution/delay_execution.c | 67 +++++++-
.../expected/cached-plan-replan.out | 156 ++++++++++++++++++
.../specs/cached-plan-replan.spec | 61 +++++++
71 files changed, 984 insertions(+), 189 deletions(-)
create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out
create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index c3ac27ae99..a0630d7944 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -78,7 +78,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
-static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool explain_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void explain_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -258,9 +258,11 @@ _PG_init(void)
/*
* ExecutorStart hook: start up logging if needed
*/
-static void
+static bool
explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
/*
* At the beginning of each top-level statement, decide whether we'll
* sample this statement. If nested-statement explaining is enabled,
@@ -296,9 +298,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
}
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
if (auto_explain_enabled())
{
@@ -316,6 +318,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 55b957d251..1160a7326a 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse,
const char *query_string,
int cursorOptions,
ParamListInfo boundParams);
-static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
+static bool pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
static void pgss_ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction,
uint64 count, bool execute_once);
@@ -963,13 +963,15 @@ pgss_planner(Query *parse,
/*
* ExecutorStart hook: start up tracking if needed
*/
-static void
+static bool
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
+
if (prev_ExecutorStart)
- prev_ExecutorStart(queryDesc, eflags);
+ plan_valid = prev_ExecutorStart(queryDesc, eflags);
else
- standard_ExecutorStart(queryDesc, eflags);
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
/*
* If query has queryId zero, don't track it. This prevents double
@@ -992,6 +994,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
MemoryContextSwitchTo(oldcxt);
}
}
+
+ return plan_valid;
}
/*
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..1edd4c3f17 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -2658,7 +2658,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags)
/* Get info about foreign table. */
rtindex = node->resultRelInfo->ri_RangeTableIndex;
if (fsplan->scan.scanrelid == 0)
+ {
dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
+ if (!ExecPlanStillValid(estate))
+ return;
+ }
else
dmstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dmstate->rel));
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index 9e4b2437a5..916d6dced3 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate,
((DR_copy *) dest)->cstate = cstate;
/* Create a QueryDesc requesting no output */
- cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ cstate->queryDesc = CreateQueryDesc(plan, NULL,
+ pstate->p_sourcetext,
GetActiveSnapshot(),
InvalidSnapshot,
dest, NULL, NULL, 0);
@@ -567,8 +568,10 @@ BeginCopyTo(ParseState *pstate,
* Call ExecutorStart to prepare the plan for execution.
*
* ExecutorStart computes a result tupdesc for us
+ *
+ * OK to ignore the return value; plan can't become invalid.
*/
- ExecutorStart(cstate->queryDesc, 0);
+ (void) ExecutorStart(cstate->queryDesc, 0);
tupDesc = cstate->queryDesc->tupDesc;
}
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e91920ca14..e5cce4c07c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -325,12 +325,16 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext,
+ queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, GetIntoRelEFlags(into));
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, GetIntoRelEFlags(into));
/* run the plan to completion */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 59d57f9c10..6171a20fe2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -416,7 +416,7 @@ ExplainOneQuery(Query *query, int cursorOptions,
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
}
- queryDesc = ExplainQueryDesc(plan, queryString, into, es,
+ queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es,
params, queryEnv);
Assert(queryDesc);
@@ -429,9 +429,11 @@ ExplainOneQuery(Query *query, int cursorOptions,
/*
* ExplainQueryDesc
* Set up QueryDesc for EXPLAINing a given plan
+ *
+ * This returns NULL if cplan is found to be no longer valid.
*/
QueryDesc *
-ExplainQueryDesc(PlannedStmt *stmt,
+ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv)
{
@@ -467,7 +469,7 @@ ExplainQueryDesc(PlannedStmt *stmt,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc for the query */
- queryDesc = CreateQueryDesc(stmt, queryString,
+ queryDesc = CreateQueryDesc(stmt, cplan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, params, queryEnv, instrument_option);
@@ -481,8 +483,18 @@ ExplainQueryDesc(PlannedStmt *stmt,
if (into)
eflags |= GetIntoRelEFlags(into);
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, eflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A cached plan
+ * may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, eflags))
+ {
+ /* Clean up. */
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ return NULL;
+ }
return queryDesc;
}
@@ -4884,6 +4896,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
}
}
+/*
+ * Discard output buffer for a fresh restart.
+ */
+void
+ExplainResetOutput(ExplainState *es)
+{
+ Assert(es->str);
+ resetStringInfo(es->str);
+ ExplainBeginOutput(es);
+}
+
/*
* Emit the start-of-output boilerplate.
*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index 4cc994ca31..8a0859a355 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -797,11 +797,13 @@ execute_sql_string(const char *sql)
QueryDesc *qdesc;
qdesc = CreateQueryDesc(stmt,
+ NULL,
sql,
GetActiveSnapshot(), NULL,
dest, NULL, NULL, 0);
- ExecutorStart(qdesc, 0);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(qdesc, 0);
ExecutorRun(qdesc, ForwardScanDirection, 0, true);
ExecutorFinish(qdesc);
ExecutorEnd(qdesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ac2e74fa3f..38795ce7ca 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -408,12 +408,16 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
UpdateActiveSnapshotCommandId();
/* Create a QueryDesc, redirecting output to our tuple receiver */
- queryDesc = CreateQueryDesc(plan, queryString,
+ queryDesc = CreateQueryDesc(plan, NULL, queryString,
GetActiveSnapshot(), InvalidSnapshot,
dest, NULL, NULL, 0);
- /* call ExecutorStart to prepare the plan for execution */
- ExecutorStart(queryDesc, 0);
+ /*
+ * call ExecutorStart to prepare the plan for execution
+ *
+ * OK to ignore the return value; plan can't become invalid.
+ */
+ (void) ExecutorStart(queryDesc, 0);
/* run the plan */
ExecutorRun(queryDesc, ForwardScanDirection, 0, true);
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index 73ed7aa2f0..5120f93414 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -142,9 +142,10 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
/*
* Start execution, inserting parameters if any.
+ *
+ * OK to ignore the return value; plan can't become invalid here.
*/
- PortalStart(portal, params, 0, GetActiveSnapshot());
-
+ (void) PortalStart(portal, params, 0, GetActiveSnapshot());
Assert(portal->strategy == PORTAL_ONE_SELECT);
/*
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 1e9a98ad6e..156c3c5fee 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate,
paramLI = EvaluateParams(pstate, entry, stmt->params, estate);
}
+replan:
/* Create a new portal to run the query in */
portal = CreateNewPortal();
/* Don't display the portal in pg_cursors, it is for internal use only */
@@ -251,9 +252,15 @@ ExecuteQuery(ParseState *pstate,
}
/*
- * Run the portal as appropriate.
+ * Run the portal as appropriate. If the portal contains a cached plan, it
+ * must be recreated if the cached plan was found to have been invalidated
+ * when initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, eflags, GetActiveSnapshot());
+ if (!PortalStart(portal, paramLI, eflags, GetActiveSnapshot()))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
(void) PortalRun(portal, count, false, true, dest, dest, qc);
@@ -574,7 +581,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
PreparedStatement *entry;
const char *query_string;
- CachedPlan *cplan;
+ CachedPlan *cplan = NULL;
List *plan_list;
ListCell *p;
ParamListInfo paramLI = NULL;
@@ -618,6 +625,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
}
/* Replan if needed, and acquire a transient refcount */
+replan:
cplan = GetCachedPlan(entry->plansource, paramLI,
CurrentResourceOwner, queryEnv);
@@ -642,9 +650,14 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
{
QueryDesc *queryDesc;
- queryDesc = ExplainQueryDesc(pstmt, queryString,
+ queryDesc = ExplainQueryDesc(pstmt, cplan, queryString,
into, es, paramLI, queryEnv);
- Assert(queryDesc != NULL);
+ if (queryDesc == NULL)
+ {
+ ExplainResetOutput(es);
+ ReleaseCachedPlan(cplan, CurrentResourceOwner);
+ goto replan;
+ }
ExplainOnePlan(queryDesc, into, es, query_string, paramLI,
queryEnv, &planduration,
(es->buffers ? &bufusage : NULL));
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 67a5c1769b..5113523bb9 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -280,6 +280,37 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
associated with ExprContexts, and commonly each PlanState node has its own
ExprContext to evaluate its qual and targetlist expressions in.
+Relation Locking
+----------------
+
+Normally, the executor does not lock non-index relations appearing in a given
+plan tree when initializing it for execution if the plan tree is freshly
+created, that is, not derived from a CachedPlan. The reason for that is that
+the locks must already have been taken during parsing, rewriting, and planning
+of the query in that case. If the plan tree is a cached one, there may still
+be unlocked relations present in the plan tree, because GetCachedPlan() only
+locks the relations that would be present in the query's range table before
+planning occurs, but not relations that would have been added to the range
+table during planning. This means that inheritance child tables present in
+a cached plan, which are added to the query's range table during planning,
+would not have been locked when the plan enters the executor.
+
+GetCachedPlan() punts on locking child tables because not all may actually be
+scanned during a given execution of the plan if the child tables are partitions
+which may get pruned away due to execution-initialization-time pruning. So the
+locking of child tables is made to wait till execution-initialization-time,
+which occurs during ExecInitNode() on the plan nodes containing the child
+tables.
+
+So, there's a time window during which a cached plan tree could go stale
+if it contains child tables, because they could get changed in other backends
+before ExecInitNode() gets a lock on them. This means the executor now must
+check the validity of the plan tree every time it takes a lock on a child
+table contained in the tree after execution-initialization-pruning has been
+performed. It does that by looking at CachedPlan.is_valid of the CachedPlan
+passed to it. If the plan tree is indeed stale (is_valid=false), the executor
+must give up continuing to initialize it any further and return to the caller
+letting it know that the execution must be retried with a new plan tree.
Query Processing Control Flow
-----------------------------
@@ -316,6 +347,12 @@ This is a sketch of control flow for full query processing:
FreeQueryDesc
+As mentioned in the "Relation Locking" section, if the plan tree is found to
+be stale during one of the recursive calls of ExecInitNode() after taking a
+lock on a child table, the control is immmediately returned to the caller of
+ExecutorStart(), which must redo the steps from CreateQueryDesc with a new
+plan tree.
+
Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 235bb52ccc..dcfbf58495 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -79,7 +79,7 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL;
ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
/* decls for local routines only used within this module */
-static void InitPlan(QueryDesc *queryDesc, int eflags);
+static bool InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
static void ExecEndPlan(EState *estate);
@@ -119,6 +119,13 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* eflags contains flag bits as described in executor.h.
*
+ * Plan initialization may fail if the input plan tree is found to have been
+ * invalidated, which can happen if it comes from a CachedPlan.
+ *
+ * Returns true if plan was successfully initialized and false otherwise. If
+ * the latter, the caller must call ExecutorEnd() on 'queryDesc' to clean up
+ * after failed plan initialization.
+ *
* NB: the CurrentMemoryContext when this is called will become the parent
* of the per-query context used for this Executor invocation.
*
@@ -128,7 +135,7 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
*
* ----------------------------------------------------------------
*/
-void
+bool
ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
@@ -140,14 +147,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
- (*ExecutorStart_hook) (queryDesc, eflags);
- else
- standard_ExecutorStart(queryDesc, eflags);
+ return (*ExecutorStart_hook) (queryDesc, eflags);
+
+ return standard_ExecutorStart(queryDesc, eflags);
}
-void
+bool
standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
{
+ bool plan_valid;
EState *estate;
MemoryContext oldcontext;
@@ -263,9 +271,11 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
/*
* Initialize the plan state tree
*/
- InitPlan(queryDesc, eflags);
+ plan_valid = InitPlan(queryDesc, eflags);
MemoryContextSwitchTo(oldcontext);
+
+ return plan_valid;
}
/* ----------------------------------------------------------------
@@ -620,6 +630,17 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos,
RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l);
Assert(OidIsValid(perminfo->relid));
+
+ /*
+ * Relations whose permissions need to be checked must already have
+ * been locked by the parser or by GetCachedPlan() if a cached plan is
+ * being executed.
+ *
+ * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan
+ * in a parallel worker?
+ */
+ Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) ||
+ IsParallelWorker());
result = ExecCheckOneRelPerms(perminfo);
if (!result)
{
@@ -829,9 +850,12 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt)
*
* Initializes the query plan: open files, allocate storage
* and start up the rule manager
+ *
+ * Returns true if the plan tree is successfully initialized for execution,
+ * false otherwise.
* ----------------------------------------------------------------
*/
-static void
+static bool
InitPlan(QueryDesc *queryDesc, int eflags)
{
CmdType operation = queryDesc->operation;
@@ -839,7 +863,7 @@ InitPlan(QueryDesc *queryDesc, int eflags)
Plan *plan = plannedstmt->planTree;
List *rangeTable = plannedstmt->rtable;
EState *estate = queryDesc->estate;
- PlanState *planstate;
+ PlanState *planstate = NULL;
TupleDesc tupType;
ListCell *l;
int i;
@@ -850,10 +874,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true);
/*
- * initialize the node's execution state
+ * Set up range table in EState.
*/
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
+ estate->es_cachedplan = queryDesc->cplan;
estate->es_plannedstmt = plannedstmt;
/*
@@ -886,6 +911,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
case ROW_MARK_KEYSHARE:
case ROW_MARK_REFERENCE:
relation = ExecGetRangeTableRelation(estate, rc->rti);
+ if (!ExecPlanStillValid(estate))
+ goto plan_init_suspended;
break;
case ROW_MARK_COPY:
/* no physical table access is required */
@@ -953,6 +980,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
sp_eflags |= EXEC_FLAG_REWIND;
subplanstate = ExecInitNode(subplan, estate, sp_eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(subplanstate == NULL);
+ goto plan_init_suspended;
+ }
estate->es_subplanstates = lappend(estate->es_subplanstates,
subplanstate);
@@ -966,6 +998,11 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* processing tuples.
*/
planstate = ExecInitNode(plan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(planstate == NULL);
+ goto plan_init_suspended;
+ }
/*
* Get the tuple descriptor describing the type of tuples to return.
@@ -1008,7 +1045,18 @@ InitPlan(QueryDesc *queryDesc, int eflags)
}
queryDesc->tupDesc = tupType;
+ Assert(planstate != NULL);
queryDesc->planstate = planstate;
+ return true;
+
+plan_init_suspended:
+ /*
+ * Plan initialization failed. Mark QueryDesc as such. ExecEndPlan()
+ * will clean up initialized plan nodes from estate->es_planstate_nodes.
+ */
+ Assert(planstate == NULL);
+ queryDesc->planstate = NULL;
+ return false;
}
/*
@@ -1426,7 +1474,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo)
/*
* All ancestors up to the root target relation must have been
- * locked by the planner or AcquireExecutorLocks().
+ * locked by the planner or ExecLockAppendNonLeafRelations().
*/
ancRel = table_open(ancOid, NoLock);
rInfo = makeNode(ResultRelInfo);
@@ -2856,7 +2904,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
* Child EPQ EStates share the parent's copy of unchanging state such as
* the snapshot, rangetable, and external Param info. They need their own
* copies of local state, including a tuple table, es_param_exec_vals,
- * result-rel info, etc.
+ * result-rel info, etc. Also, we don't pass the parent't copy of the
+ * CachedPlan, because no new locks will be taken for EvalPlanQual().
*/
rcestate->es_direction = ForwardScanDirection;
rcestate->es_snapshot = parentestate->es_snapshot;
@@ -2943,6 +2992,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
PlanState *subplanstate;
subplanstate = ExecInitNode(subplan, rcestate, 0);
+
+ /*
+ * At this point, we had better not received any new invalidation
+ * messages that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && subplanstate);
rcestate->es_subplanstates = lappend(rcestate->es_subplanstates,
subplanstate);
}
@@ -2986,6 +3041,12 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
*/
epqstate->recheckplanstate = ExecInitNode(planTree, rcestate, 0);
+ /*
+ * At this point, we had better not received any new invalidation messages
+ * that would have caused the plan tree to go stale.
+ */
+ Assert(ExecPlanStillValid(rcestate) && epqstate->recheckplanstate);
+
MemoryContextSwitchTo(oldcontext);
}
@@ -3008,6 +3069,10 @@ EvalPlanQualEnd(EPQState *epqstate)
MemoryContext oldcontext;
ListCell *l;
+ /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */
+ if (epqstate->parentestate == NULL)
+ return;
+
rtsize = epqstate->parentestate->es_range_table_size;
/*
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cc2b8ccab7..bfa2a8ec18 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1248,8 +1248,17 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver,
paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false);
paramLI = RestoreParamList(¶mspace);
- /* Create a QueryDesc for the query. */
+ /*
+ * Create a QueryDesc for the query. Note that no CachedPlan is available
+ * here even if the leader may have gotten the plan tree from one. That's
+ * fine though, because the leader would have taken the locks necessary
+ * for the plan tree that we have here to be fully valid. That is true
+ * despite the fact that we will be taking our own copies of those locks
+ * in ExecGetRangeTableRelation(), because none of them would be the locks
+ * that are not already taken by the leader.
+ */
return CreateQueryDesc(pstmt,
+ NULL,
queryString,
GetActiveSnapshot(), InvalidSnapshot,
receiver, paramLI, NULL, instrument_options);
@@ -1430,7 +1439,8 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Start up the executor */
queryDesc->plannedstmt->jitFlags = fpes->jit_flags;
- ExecutorStart(queryDesc, fpes->eflags);
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(queryDesc, fpes->eflags);
/* Special executor initialization steps for parallel workers */
queryDesc->planstate->state->es_query_dsa = area;
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index eb8a87fd63..cf73d28baa 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -513,6 +513,13 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
oldcxt = MemoryContextSwitchTo(proute->memcxt);
+ /*
+ * Note that while we normally check ExecPlanStillValid(estate) after each
+ * lock taken during execution initialization, it is fine not do so for
+ * partitions opened here, for tuple routing. Locks taken here can't
+ * possibly invalidate the plan given that the plan doesn't contain any
+ * info about those partitions.
+ */
partrel = table_open(partOid, RowExclusiveLock);
leaf_part_rri = makeNode(ResultRelInfo);
@@ -1111,6 +1118,9 @@ ExecInitPartitionDispatchInfo(EState *estate,
* Only sub-partitioned tables need to be locked here. The root
* partitioned table will already have been locked as it's referenced in
* the query's rtable.
+ *
+ * See the comment in ExecInitPartitionInfo() about taking locks and
+ * not checking ExecPlanStillValid(estate) here.
*/
if (partoid != RelationGetRelid(proute->partition_root))
rel = table_open(partoid, RowExclusiveLock);
@@ -1801,6 +1811,8 @@ ExecInitPartitionPruning(PlanState *planstate,
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Perform an initial partition prune pass, if required.
@@ -1927,6 +1939,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
* duration of this executor run.
*/
partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
partkey = RelationGetPartitionKey(partrel);
partdesc = PartitionDirectoryLookup(estate->es_partition_directory,
partrel);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 653f74cf58..2dcacafd03 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -135,10 +135,17 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'estate' is the shared execution state for the plan tree
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
- * Returns a PlanState node corresponding to the given Plan node.
+ * Returns a PlanState node corresponding to the given Plan node or NULL.
*
- * As a side-effect, all PlanState nodes that are created are appended to
- * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
+ * NULL may be returned either if the input node is NULL or if the plan
+ * tree that the node is a part of is found to have been invalidated when
+ * taking a lock on the relation mentioned in the node or in a child
+ * node. The latter case arises if the plan tree contains inheritance/
+ * partition child tables and is from a CachedPlan.
+ *
+ * As a side-effect, all PlanState nodes that are successfully created are
+ * appended to estate->es_planstate_nodes for the cleanup processing in
+ * ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -391,6 +398,13 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
break;
}
+ if (!ExecPlanStillValid(estate))
+ {
+ Assert(result == NULL);
+ return NULL;
+ }
+
+ Assert(result != NULL);
ExecSetExecProcNode(result, result->ExecProcNode);
/*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index b567165003..ee12235b2f 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -806,7 +806,25 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rtekind == RTE_RELATION);
- if (!IsParallelWorker())
+ if (IsParallelWorker() ||
+ (estate->es_cachedplan != NULL && !rte->inFromCl))
+ {
+ /*
+ * Take a lock if we are a parallel worker or if this is a child
+ * table referenced in a cached plan.
+ *
+ * Parallel workers need to have their own local lock on the
+ * relation. This ensures sane behavior in case the parent process
+ * exits before we do.
+ *
+ * When executing a cached plan, child tables must be locked
+ * here, because plancache.c (GetCachedPlan()) would only have
+ * locked tables mentioned in the query, that is, tables whose
+ * RTEs' inFromCl is true.
+ */
+ rel = table_open(rte->relid, rte->rellockmode);
+ }
+ else
{
/*
* In a normal query, we should already have the appropriate lock,
@@ -819,15 +837,6 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
Assert(rte->rellockmode == AccessShareLock ||
CheckRelationLockedByMe(rel, rte->rellockmode, false));
}
- else
- {
- /*
- * If we are a parallel worker, we need to obtain our own local
- * lock on the relation. This ensures sane behavior in case the
- * parent process exits before we do.
- */
- rel = table_open(rte->relid, rte->rellockmode);
- }
estate->es_relations[rti - 1] = rel;
}
@@ -835,6 +844,38 @@ ExecGetRangeTableRelation(EState *estate, Index rti)
return rel;
}
+/*
+ * ExecLockAppendNonLeafRelations
+ * Lock non-leaf relations whose children are scanned by a given
+ * Append/MergeAppend node
+ */
+void
+ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids)
+{
+ ListCell *l;
+
+ /* This should get called only when executing cached plans. */
+ Assert(estate->es_cachedplan != NULL);
+ foreach(l, allpartrelids)
+ {
+ Bitmapset *partrelids = lfirst_node(Bitmapset, l);
+ int i;
+
+ /*
+ * Note that we don't lock the first member (i=0) of each bitmapset
+ * because it stands for the root parent mentioned in the query that
+ * should always have been locked before entering the executor.
+ */
+ i = 0;
+ while ((i = bms_next_member(partrelids, i)) > 0)
+ {
+ RangeTblEntry *rte = exec_rt_fetch(i, estate);
+
+ LockRelationOid(rte->relid, rte->rellockmode);
+ }
+ }
+}
+
/*
* ExecInitResultRelation
* Open relation given by the passed-in RT index and fill its
@@ -850,6 +891,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Relation resultRelationDesc;
resultRelationDesc = ExecGetRangeTableRelation(estate, rti);
+ if (!ExecPlanStillValid(estate))
+ return;
InitResultRelInfo(resultRelInfo,
resultRelationDesc,
rti,
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index f55424eb5a..4ddf4fd7a9 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
dest = None_Receiver;
es->qd = CreateQueryDesc(es->stmt,
+ NULL, /* fmgr_sql() doesn't use CachedPlans */
fcache->src,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -862,7 +863,9 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
eflags = EXEC_FLAG_SKIP_TRIGGERS;
else
eflags = 0; /* default run-to-completion flags */
- ExecutorStart(es->qd, eflags);
+
+ /* OK to ignore the return value; plan can't become invalid. */
+ (void) ExecutorStart(es->qd, eflags);
}
es->status = F_EXEC_RUN;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index e9d9ab6bdd..9553a85115 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -3304,6 +3304,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
eflags &= ~EXEC_FLAG_REWIND;
outerPlan = outerPlan(node);
outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type.
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 9148d7d3b1..222434a84d 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -133,6 +133,25 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
appendstate->as_syncdone = false;
appendstate->as_begun = false;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->appendplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -147,6 +166,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -221,6 +242,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
firstvalid = j;
appendplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
appendstate->as_first_partial_plan = firstvalid;
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 147592f7e2..53afcef21c 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -88,8 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d58ee4f4e1..388a02ec99 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -760,11 +760,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child nodes
*/
outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* get the scan type from the relation descriptor.
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 83ec9ede89..99015812a1 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -211,6 +211,7 @@ BitmapIndexScanState *
ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
{
BitmapIndexScanState *indexstate;
+ Relation indexRelation;
LOCKMODE lockmode;
/* check for unsupported flags */
@@ -262,7 +263,13 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->biss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->biss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 736852a0ae..425f22ee48 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -89,8 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags)
foreach(l, node->bitmapplans)
{
initNode = (Plan *) lfirst(l);
- bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags);
- i++;
+ bitmapplanstates[i++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
/*
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index bd42c65b29..91239cc500 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
if (scanrelid > 0)
{
scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
css->ss.ss_currentRelation = scan_rel;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index e6616dd718..71495313db 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (scanrelid > 0)
{
currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
scanstate->ss.ss_currentRelation = currentRelation;
fdwroutine = GetFdwRoutineForRelation(currentRelation, true);
}
@@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
if (outerPlan(node))
outerPlanState(scanstate) =
ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Tell the FDW to initialize the scan.
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index f7a69f185b..c5652aeb2d 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -89,6 +89,9 @@ ExecInitGather(Gather *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
tupDesc = ExecGetResultType(outerPlanState(gatherstate));
/*
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index d357ff0c47..1191b9e420 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags)
*/
outerNode = outerPlan(node);
outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Leader may access ExecProcNode result directly (if
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 2badcc7e60..b4c3044c1f 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index edd2324384..b2119febb6 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize our result slot and type. No need to build projection
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 8078d7f229..d5ff80660e 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -752,8 +752,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
hashNode = (Hash *) innerPlan(node);
outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(hjstate));
innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(hjstate));
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 52b146cfb8..785896e5ea 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags)
* nodes may be able to do something more useful.
*/
outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 0b43a9b969..ea8bef4b97 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -490,6 +490,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
IndexOnlyScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
TupleDesc tupDesc;
@@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -564,7 +567,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->ioss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 4540c7781d..956e9e5543 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -904,6 +904,7 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
{
IndexScanState *indexstate;
Relation currentRelation;
+ Relation indexRelation;
LOCKMODE lockmode;
/*
@@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
indexstate->ss.ss_currentRelation = currentRelation;
indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
@@ -969,7 +972,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
/* Open the index relation. */
lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode;
- indexstate->iss_RelationDesc = index_open(node->indexid, lockmode);
+ indexRelation = index_open(node->indexid, lockmode);
+ if (!ExecPlanStillValid(estate))
+ {
+ index_close(indexRelation, lockmode);
+ return NULL;
+ }
+ indexstate->iss_RelationDesc = indexRelation;
/*
* Initialize index-specific scan state
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index a75099dd73..a1fc36a3f0 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 55de8d3d65..ff86a82b92 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -322,6 +322,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* node returns unmodified slots from the outer plan */
lrstate->ps.resultopsset = true;
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index ef04e9a8e7..8d02ac0ccb 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags)
outerPlan = outerPlan(node);
outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result type and slot. No need to initialize projection info
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 61578d4b5c..a994d48fb2 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags)
outerNode = outerPlan(node);
outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize return slot and type. No need to initialize projection info
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 8aa64944c9..14d07c30e8 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -81,6 +81,25 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
mergestate->ps.state = estate;
mergestate->ps.ExecProcNode = ExecMergeAppend;
+ /*
+ * Lock non-leaf partitions whose leaf children are present in
+ * node->mergeplans. Only need to do so if executing a cached
+ * plan, because child tables present in cached plans are not
+ * locked before execution.
+ *
+ * XXX - some of the non-leaf partitions may also be mentioned in
+ * part_prune_info, which if they are would get locked again in
+ * ExecInitPartitionPruning() because it calls
+ * ExecGetRangeTableRelation() which locks child tables.
+ */
+ if (estate->es_cachedplan)
+ {
+ ExecLockAppendNonLeafRelations(estate, node->allpartrelids);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
+ }
+
/* If run-time partition pruning is enabled, then set that up now */
if (node->part_prune_info != NULL)
{
@@ -95,6 +114,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
@@ -151,6 +172,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
Plan *initNode = (Plan *) list_nth(node->mergeplans, i);
mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
}
mergestate->ps.ps_ProjInfo = NULL;
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 7b530d9088..0d92ec278a 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1490,11 +1490,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
mergestate->mj_SkipMarkRestore = node->skip_mark_restore;
outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(mergestate));
innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate,
mergestate->mj_SkipMarkRestore ?
eflags :
(eflags | EXEC_FLAG_MARK));
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerDesc = ExecGetResultType(innerPlanState(mergestate));
/*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index bdbaa4753b..2245a67397 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -3984,6 +3984,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
linitial_int(node->resultRelations));
}
+ if (!ExecPlanStillValid(estate))
+ return NULL;
+
/* set up epqstate with dummy subplan data for the moment */
EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL,
node->epqParam, node->resultRelations);
@@ -4011,6 +4014,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (resultRelInfo != mtstate->rootResultRelInfo)
{
ExecInitResultRelation(estate, resultRelInfo, resultRelation);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* For child result relations, store the root result relation
@@ -4038,6 +4043,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
* Now we may initialize the subplan.
*/
outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Do additional per-result-relation initialization.
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index 5cfb50a366..e24554f4f8 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags)
* values.
*/
outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
if (node->nestParams == NIL)
eflags |= EXEC_FLAG_REWIND;
else
eflags &= ~EXEC_FLAG_REWIND;
innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index 4a388220ee..863bf2cc65 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index aee31c7139..3f3de771d0 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* If hashing, precompute fmgr lookup data for inner loop, and create the
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index a100b144be..f2206af451 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* initialize child nodes
*/
outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* we don't use inner plan
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index d7e22b1dbb..22357e7a0e 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* we won't set up the HeapScanDesc till later */
scanstate->ss.ss_currentScanDesc = NULL;
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4da0f28f7b..b0b34cd14e 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
ExecOpenScanRelation(estate,
node->scan.scanrelid,
eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index f7db9a3415..3535aa298c 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags)
if (node->strategy == SETOP_HASHED)
eflags &= ~EXEC_FLAG_REWIND;
outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
outerDesc = ExecGetResultType(outerPlanState(setopstate));
/*
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index 078d041c40..547203ebfd 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK);
outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index bc55a82fc3..8b6629c939 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
* initialize subquery
*/
subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize scan slot and type (needed by ExecAssignScanProjectionInfo)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 2124c55ef5..613b377c7c 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidrangestate->ss.ss_currentRelation = currentRelation;
tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 862bd0330b..1b0a2d8083 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
* open the scan relation
*/
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
tidstate->ss.ss_currentRelation = currentRelation;
tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 50babacdc8..ae9af8f21e 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags)
* then initialize outer plan
*/
outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* Initialize result slot and type. Unique nodes do no projections, so
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 648cdadc32..77de2d0c22 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2458,6 +2458,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
*/
outerPlan = outerPlan(node);
outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags);
+ if (!ExecPlanStillValid(estate))
+ return NULL;
/*
* initialize source tuple type (which is also the tuple type that we'll
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index d36ca35d3a..9c4ed74240 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1582,6 +1582,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
Snapshot snapshot;
MemoryContext oldcontext;
Portal portal;
+ bool plan_valid;
SPICallbackArg spicallbackarg;
ErrorContextCallback spierrcontext;
@@ -1623,6 +1624,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
_SPI_current->processed = 0;
_SPI_current->tuptable = NULL;
+replan:
/* Create the portal */
if (name == NULL || name[0] == '\0')
{
@@ -1766,15 +1768,23 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan,
}
/*
- * Start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, paramLI, 0, snapshot);
+ plan_valid = PortalStart(portal, paramLI, 0, snapshot);
Assert(portal->strategy != PORTAL_MULTI_QUERY);
/* Pop the error context stack */
error_context_stack = spierrcontext.previous;
+ if (!plan_valid)
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
+
/* Pop the SPI stack */
_SPI_end_call(true);
@@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
* Replan if needed, and increment plan refcount. If it's a saved
* plan, the refcount must be backed by the plan_owner.
*/
+replan:
cplan = GetCachedPlan(plansource, options->params,
plan_owner, _SPI_current->queryEnv);
@@ -2669,6 +2680,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
snap = InvalidSnapshot;
qdesc = CreateQueryDesc(stmt,
+ cplan,
plansource->query_string,
snap, crosscheck_snapshot,
dest,
@@ -2682,10 +2694,16 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
else
eflags = EXEC_FLAG_SKIP_TRIGGERS;
- ExecutorStart(qdesc, eflags);
+ if (!ExecutorStart(qdesc, eflags))
+ {
+ ExecutorEnd(qdesc);
+ FreeQueryDesc(qdesc);
+ Assert(cplan);
+ ReleaseCachedPlan(cplan, plan_owner);
+ goto replan;
+ }
res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0);
-
FreeQueryDesc(qdesc);
}
else
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index ee9b89a672..c807e9cdcc 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -27,6 +27,7 @@
#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/inval.h"
+#include "utils/lsyscache.h"
/*
@@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger)
return false;
}
+/*
+ * CheckRelLockedByMe
+ *
+ * Returns true if current transaction holds a lock on the given relation of
+ * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK.
+ * ("Stronger" is defined as "numerically higher", which is a bit
+ * semantically dubious but is OK for the purposes we use this for.)
+ */
+bool
+CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger)
+{
+ Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId;
+ LOCKTAG tag;
+
+ SET_LOCKTAG_RELATION(tag, dbId, relid);
+
+ if (LockHeldByMe(&tag, lockmode))
+ return true;
+
+ if (orstronger)
+ {
+ LOCKMODE slockmode;
+
+ for (slockmode = lockmode + 1;
+ slockmode <= MaxLockMode;
+ slockmode++)
+ {
+ if (LockHeldByMe(&tag, slockmode))
+ {
+#ifdef NOT_USED
+ /* Sometimes this might be useful for debugging purposes */
+ elog(WARNING, "lock mode %s substituted for %s on relation %s",
+ GetLockmodeName(tag.locktag_lockmethodid, slockmode),
+ GetLockmodeName(tag.locktag_lockmethodid, lockmode),
+ RelationGetRelationName(relation));
+#endif
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
/*
* LockHasWaitersRelation
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 36cc99ec9c..88724a8d67 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1232,7 +1232,12 @@ exec_simple_query(const char *query_string)
/*
* Start the portal. No parameters here.
*/
- PortalStart(portal, NULL, 0, InvalidSnapshot);
+ {
+ bool plan_valid PG_USED_FOR_ASSERTS_ONLY;
+
+ plan_valid = PortalStart(portal, NULL, 0, InvalidSnapshot);
+ Assert(plan_valid);
+ }
/*
* Select the appropriate output format: text unless we are doing a
@@ -1737,6 +1742,7 @@ exec_bind_message(StringInfo input_message)
"commands ignored until end of transaction block"),
errdetail_abort()));
+replan:
/*
* Create the portal. Allow silent replacement of an existing portal only
* if the unnamed portal is specified.
@@ -2028,9 +2034,15 @@ exec_bind_message(StringInfo input_message)
PopActiveSnapshot();
/*
- * And we're ready to start portal execution.
+ * Start portal execution. If the portal contains a cached plan, it must
+ * be recreated if the cached plan was found to have been invalidated when
+ * initializing one of the plan trees contained in it.
*/
- PortalStart(portal, params, 0, InvalidSnapshot);
+ if (!PortalStart(portal, params, 0, InvalidSnapshot))
+ {
+ PortalDrop(portal, false);
+ goto replan;
+ }
/*
* Apply the result format requests to the portal.
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 701808f303..48cd6f4304 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -60,6 +60,7 @@ static void DoPortalRewind(Portal portal);
*/
QueryDesc *
CreateQueryDesc(PlannedStmt *plannedstmt,
+ CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
@@ -72,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt,
qd->operation = plannedstmt->commandType; /* operation */
qd->plannedstmt = plannedstmt; /* plan */
+ qd->cplan = cplan; /* CachedPlan, if plan is from one */
qd->sourceText = sourceText; /* query text */
qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */
/* RI check snapshot */
@@ -341,10 +343,12 @@ FetchStatementTargetList(Node *stmt)
* presently ignored for non-PORTAL_ONE_SELECT portals (it's only intended
* to be used for cursors).
*
- * On return, portal is ready to accept PortalRun() calls, and the result
- * tupdesc (if any) is known.
+ * True is returned if portal is ready to accept PortalRun() calls, and the
+ * result tupdesc (if any) is known. False if the plan tree is no longer
+ * valid, in which case, the caller must retry after generating a new
+ * CachedPlan.
*/
-void
+bool
PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot)
{
@@ -353,6 +357,7 @@ PortalStart(Portal portal, ParamListInfo params,
MemoryContext oldContext;
QueryDesc *queryDesc;
int myeflags = 0;
+ bool plan_valid = true;
Assert(PortalIsValid(portal));
Assert(portal->status == PORTAL_DEFINED);
@@ -407,6 +412,7 @@ PortalStart(Portal portal, ParamListInfo params,
* set the destination to DestNone.
*/
queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts),
+ portal->cplan,
portal->sourceText,
GetActiveSnapshot(),
InvalidSnapshot,
@@ -431,8 +437,19 @@ PortalStart(Portal portal, ParamListInfo params,
else
myeflags = eflags;
- /* Call ExecutorStart to prepare the plan for execution. */
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for execution. A
+ * cached plan may get invalidated during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ PopActiveSnapshot();
+ plan_valid = false;
+ goto plan_init_failed;
+ }
/*
* This tells PortalCleanup to shut down the executor, though
@@ -525,7 +542,7 @@ PortalStart(Portal portal, ParamListInfo params,
* Create the QueryDesc. DestReceiver will be set in
* PortalRunMulti() before calling ExecutorRun().
*/
- queryDesc = CreateQueryDesc(plan,
+ queryDesc = CreateQueryDesc(plan, portal->cplan,
portal->sourceText,
!is_utility ?
GetActiveSnapshot() :
@@ -541,7 +558,20 @@ PortalStart(Portal portal, ParamListInfo params,
if (is_utility)
continue;
- ExecutorStart(queryDesc, myeflags);
+ /*
+ * Call ExecutorStart to prepare the plan for
+ * execution. A cached plan may get invalidated
+ * during plan intialization.
+ */
+ if (!ExecutorStart(queryDesc, myeflags))
+ {
+ PopActiveSnapshot();
+ Assert(queryDesc->cplan);
+ ExecutorEnd(queryDesc);
+ FreeQueryDesc(queryDesc);
+ plan_valid = false;
+ goto plan_init_failed;
+ }
PopActiveSnapshot();
}
}
@@ -563,12 +593,15 @@ PortalStart(Portal portal, ParamListInfo params,
}
PG_END_TRY();
+ portal->status = PORTAL_READY;
+
+plan_init_failed:
MemoryContextSwitchTo(oldContext);
ActivePortal = saveActivePortal;
CurrentResourceOwner = saveResourceOwner;
- portal->status = PORTAL_READY;
+ return plan_valid;
}
/*
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index fc6d267e44..2725d02312 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2095,6 +2095,27 @@ get_rel_persistence(Oid relid)
return result;
}
+/*
+ * get_rel_relisshared
+ *
+ * Returns if the given relation is shared or not
+ */
+bool
+get_rel_relisshared(Oid relid)
+{
+ HeapTuple tp;
+ Form_pg_class reltup;
+ bool result;
+
+ tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for relation %u", relid);
+ reltup = (Form_pg_class) GETSTRUCT(tp);
+ result = reltup->relisshared;
+ ReleaseSysCache(tp);
+
+ return result;
+}
/* ---------- TRANSFORM CACHE ---------- */
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index d67cd9a405..c5a7616b33 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -102,13 +102,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource);
static List *RevalidateCachedQuery(CachedPlanSource *plansource,
QueryEnvironment *queryEnv);
static bool CheckCachedPlan(CachedPlanSource *plansource);
+static bool GenericPlanIsValid(CachedPlan *cplan);
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
ParamListInfo boundParams, QueryEnvironment *queryEnv);
static bool choose_custom_plan(CachedPlanSource *plansource,
ParamListInfo boundParams);
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
static Query *QueryListGetPrimaryStmt(List *stmts);
-static void AcquireExecutorLocks(List *stmt_list, bool acquire);
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
static void ScanQueryForLocks(Query *parsetree, bool acquire);
static bool ScanQueryWalker(Node *node, bool *acquire);
@@ -790,8 +790,15 @@ RevalidateCachedQuery(CachedPlanSource *plansource,
* Caller must have already called RevalidateCachedQuery to verify that the
* querytree is up to date.
*
- * On a "true" return, we have acquired the locks needed to run the plan.
- * (We must do this for the "true" result to be race-condition-free.)
+ * If the plan contains any child relations that would have been added by the
+ * planner, they would not have been locked yet, because AcquirePlannerLocks()
+ * only locks relations that would be present in the original query's range
+ * table (that is, before entering the planner). So, the plan could go stale
+ * before it reaches execution if any of those child relations get modified
+ * concurrently. The executor must check that the plan (CachedPlan) is still
+ * valid after taking a lock on each of the child tables during the plan
+ * initialization phase, and if it is not, ask the caller to recreate the
+ * plan.
*/
static bool
CheckCachedPlan(CachedPlanSource *plansource)
@@ -805,60 +812,56 @@ CheckCachedPlan(CachedPlanSource *plansource)
if (!plan)
return false;
- Assert(plan->magic == CACHEDPLAN_MAGIC);
- /* Generic plans are never one-shot */
- Assert(!plan->is_oneshot);
+ if (GenericPlanIsValid(plan))
+ return true;
/*
- * If plan isn't valid for current role, we can't use it.
+ * Plan has been invalidated, so unlink it from the parent and release it.
*/
- if (plan->is_valid && plan->dependsOnRole &&
- plan->planRoleId != GetUserId())
- plan->is_valid = false;
+ ReleaseGenericPlan(plansource);
- /*
- * If it appears valid, acquire locks and recheck; this is much the same
- * logic as in RevalidateCachedQuery, but for a plan.
- */
- if (plan->is_valid)
+ return false;
+}
+
+/*
+ * GenericPlanIsValid
+ * Is a generic plan still valid?
+ *
+ * It may have gone stale due to concurrent schema modifications of relations
+ * mentioned in the plan or a couple of other things mentioned below.
+ */
+static bool
+GenericPlanIsValid(CachedPlan *cplan)
+{
+ Assert(cplan != NULL);
+ Assert(cplan->magic == CACHEDPLAN_MAGIC);
+ /* Generic plans are never one-shot */
+ Assert(!cplan->is_oneshot);
+
+ if (cplan->is_valid)
{
/*
* Plan must have positive refcount because it is referenced by
* plansource; so no need to fear it disappears under us here.
*/
- Assert(plan->refcount > 0);
-
- AcquireExecutorLocks(plan->stmt_list, true);
+ Assert(cplan->refcount > 0);
/*
- * If plan was transient, check to see if TransactionXmin has
- * advanced, and if so invalidate it.
+ * If plan isn't valid for current role, we can't use it.
*/
- if (plan->is_valid &&
- TransactionIdIsValid(plan->saved_xmin) &&
- !TransactionIdEquals(plan->saved_xmin, TransactionXmin))
- plan->is_valid = false;
+ if (cplan->dependsOnRole && cplan->planRoleId != GetUserId())
+ cplan->is_valid = false;
/*
- * By now, if any invalidation has happened, the inval callback
- * functions will have marked the plan invalid.
+ * If plan was transient, check to see if TransactionXmin has
+ * advanced, and if so invalidate it.
*/
- if (plan->is_valid)
- {
- /* Successfully revalidated and locked the query. */
- return true;
- }
-
- /* Oops, the race case happened. Release useless locks. */
- AcquireExecutorLocks(plan->stmt_list, false);
+ if (TransactionIdIsValid(cplan->saved_xmin) &&
+ !TransactionIdEquals(cplan->saved_xmin, TransactionXmin))
+ cplan->is_valid = false;
}
- /*
- * Plan has been invalidated, so unlink it from the parent and release it.
- */
- ReleaseGenericPlan(plansource);
-
- return false;
+ return cplan->is_valid;
}
/*
@@ -1128,8 +1131,16 @@ cached_plan_cost(CachedPlan *plan, bool include_planner)
* plan or a custom plan for the given parameters: the caller does not know
* which it will get.
*
- * On return, the plan is valid and we have sufficient locks to begin
- * execution.
+ * On return, the plan is valid unless it contains inheritance/partition child
+ * tables, because they will not have been locked as here we only lock the
+ * tables mentioned in the original query. Inheritance/partition child tables
+ * are locked by the executor when initializing the plan tree and if the plan
+ * gets invalidated as a result of taking those locks, the executor must ask
+ * the caller to get a new plan by calling here again. Locking of the child
+ * tables is deferred to the executor in this manner, because not all child
+ * tables may need to be locked as some may get pruned during the executor
+ * plan initialization which performs initial pruing on any nodes that
+ * support partition pruning.
*
* On return, the refcount of the plan has been incremented; a later
* ReleaseCachedPlan() call is expected. If "owner" is not NULL then
@@ -1164,7 +1175,10 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
{
if (CheckCachedPlan(plansource))
{
- /* We want a generic plan, and we already have a valid one */
+ /*
+ * We want a generic plan, and we already have a valid one, though
+ * see the header comment.
+ */
plan = plansource->gplan;
Assert(plan->magic == CACHEDPLAN_MAGIC);
}
@@ -1362,8 +1376,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
}
/*
- * Reject if AcquireExecutorLocks would have anything to do. This is
- * probably unnecessary given the previous check, but let's be safe.
+ * Reject if the executor would need to take additional locks, that is, in
+ * addition to those taken by AcquirePlannerLocks() on a given query.
*/
foreach(lc, plan->stmt_list)
{
@@ -1739,58 +1753,6 @@ QueryListGetPrimaryStmt(List *stmts)
return NULL;
}
-/*
- * AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
- * or release them if acquire is false.
- */
-static void
-AcquireExecutorLocks(List *stmt_list, bool acquire)
-{
- ListCell *lc1;
-
- foreach(lc1, stmt_list)
- {
- PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1);
- ListCell *lc2;
-
- if (plannedstmt->commandType == CMD_UTILITY)
- {
- /*
- * Ignore utility statements, except those (such as EXPLAIN) that
- * contain a parsed-but-not-planned query. Note: it's okay to use
- * ScanQueryForLocks, even though the query hasn't been through
- * rule rewriting, because rewriting doesn't change the query
- * representation.
- */
- Query *query = UtilityContainsQuery(plannedstmt->utilityStmt);
-
- if (query)
- ScanQueryForLocks(query, acquire);
- continue;
- }
-
- foreach(lc2, plannedstmt->rtable)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
-
- if (!(rte->rtekind == RTE_RELATION ||
- (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid))))
- continue;
-
- /*
- * Acquire the appropriate type of lock on each relation OID. Note
- * that we don't actually try to open the rel, and hence will not
- * fail if it's been dropped entirely --- we'll just transiently
- * acquire a non-conflicting lock.
- */
- if (acquire)
- LockRelationOid(rte->relid, rte->rellockmode);
- else
- UnlockRelationOid(rte->relid, rte->rellockmode);
- }
- }
-}
-
/*
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
* or release them if acquire is false.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 08ea852b65..392abb5150 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -88,7 +88,7 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv);
-extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt,
+extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan,
const char *queryString, IntoClause *into, ExplainState *es,
ParamListInfo params, QueryEnvironment *queryEnv);
extern void ExplainOnePlan(QueryDesc *queryDesc,
@@ -108,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m
extern void ExplainBeginOutput(ExplainState *es);
extern void ExplainEndOutput(ExplainState *es);
+extern void ExplainResetOutput(ExplainState *es);
extern void ExplainSeparatePlans(ExplainState *es);
extern void ExplainPropertyList(const char *qlabel, List *data,
diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h
index af2bf36dfb..4b7368a0dc 100644
--- a/src/include/executor/execdesc.h
+++ b/src/include/executor/execdesc.h
@@ -32,9 +32,12 @@
*/
typedef struct QueryDesc
{
+ NodeTag type;
+
/* These fields are provided by CreateQueryDesc */
CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */
PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */
+ struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */
const char *sourceText; /* source text of the query */
Snapshot snapshot; /* snapshot to use for query */
Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */
@@ -57,6 +60,7 @@ typedef struct QueryDesc
/* in pquery.c */
extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt,
+ struct CachedPlan *cplan,
const char *sourceText,
Snapshot snapshot,
Snapshot crosscheck_snapshot,
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index c677e490d7..edf2f13d04 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -19,6 +19,7 @@
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
+#include "utils/plancache.h"
/*
@@ -72,7 +73,7 @@
/* Hook for plugins to get control in ExecutorStart() */
-typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
+typedef bool (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags);
extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook;
/* Hook for plugins to get control in ExecutorRun() */
@@ -197,8 +198,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull)
/*
* prototypes from functions in execMain.c
*/
-extern void ExecutorStart(QueryDesc *queryDesc, int eflags);
-extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool ExecutorStart(QueryDesc *queryDesc, int eflags);
+extern bool standard_ExecutorStart(QueryDesc *queryDesc, int eflags);
extern void ExecutorRun(QueryDesc *queryDesc,
ScanDirection direction, uint64 count, bool execute_once);
extern void standard_ExecutorRun(QueryDesc *queryDesc,
@@ -256,6 +257,17 @@ extern void ExecEndNode(PlanState *node);
extern void ExecShutdownNode(PlanState *node);
extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node);
+/*
+ * Is the cached plan, if any, still valid at this point? That is, not
+ * invalidated by the incoming invalidation messages that have been processed
+ * recently.
+ */
+static inline bool
+ExecPlanStillValid(EState *estate)
+{
+ return estate->es_cachedplan == NULL ? true :
+ CachedPlanStillValid(estate->es_cachedplan);
+}
/* ----------------------------------------------------------------
* ExecProcNode
@@ -590,6 +602,7 @@ exec_rt_fetch(Index rti, EState *estate)
}
extern Relation ExecGetRangeTableRelation(EState *estate, Index rti);
+extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids);
extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo,
Index rti);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 233fb6b4f9..20c1bacae1 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -623,6 +623,8 @@ typedef struct EState
* ExecRowMarks, or NULL if none */
List *es_rteperminfos; /* List of RTEPermissionInfo */
PlannedStmt *es_plannedstmt; /* link to top of plan tree */
+ struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from
+ * one */
const char *es_sourceText; /* Source text from QueryDesc */
JunkFilter *es_junkFilter; /* top-level junk filter, if any */
diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h
index 4ee91e3cf9..598bf2688a 100644
--- a/src/include/storage/lmgr.h
+++ b/src/include/storage/lmgr.h
@@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode);
extern void UnlockRelation(Relation relation, LOCKMODE lockmode);
extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode,
bool orstronger);
+extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger);
extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode);
extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode);
diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h
index a5e65b98aa..577b81a9ee 100644
--- a/src/include/tcop/pquery.h
+++ b/src/include/tcop/pquery.h
@@ -29,7 +29,7 @@ extern List *FetchPortalTargetList(Portal portal);
extern List *FetchStatementTargetList(Node *stmt);
-extern void PortalStart(Portal portal, ParamListInfo params,
+extern bool PortalStart(Portal portal, ParamListInfo params,
int eflags, Snapshot snapshot);
extern void PortalSetResultFormat(Portal portal, int nFormats,
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index f5fdbfe116..a024e5dcd0 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -140,6 +140,7 @@ extern char get_rel_relkind(Oid relid);
extern bool get_rel_relispartition(Oid relid);
extern Oid get_rel_tablespace(Oid relid);
extern char get_rel_persistence(Oid relid);
+extern bool get_rel_relisshared(Oid relid);
extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes);
extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes);
extern bool get_typisdefined(Oid typid);
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 916e59d9fe..c83a67fea3 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource,
ParamListInfo boundParams,
ResourceOwner owner,
QueryEnvironment *queryEnv);
+
+/*
+ * CachedPlanStillValid
+ * Returns if a cached generic plan is still valid
+ *
+ * Called by the executor on every relation lock taken when initializing the
+ * plan tree in the CachedPlan.
+ */
+static inline bool
+CachedPlanStillValid(CachedPlan *cplan)
+{
+ return cplan->is_valid;
+}
+
extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner);
extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile
index 70f24e846d..2fca84d027 100644
--- a/src/test/modules/delay_execution/Makefile
+++ b/src/test/modules/delay_execution/Makefile
@@ -8,7 +8,8 @@ OBJS = \
delay_execution.o
ISOLATION = partition-addition \
- partition-removal-1
+ partition-removal-1 \
+ cached-plan-replan
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c
index 7cd76eb34b..ce189156ad 100644
--- a/src/test/modules/delay_execution/delay_execution.c
+++ b/src/test/modules/delay_execution/delay_execution.c
@@ -1,14 +1,18 @@
/*-------------------------------------------------------------------------
*
* delay_execution.c
- * Test module to allow delay between parsing and execution of a query.
+ * Test module to introduce delay at various points during execution of a
+ * query to test that execution proceeds safely in light of concurrent
+ * changes.
*
* The delay is implemented by taking and immediately releasing a specified
* advisory lock. If another process has previously taken that lock, the
* current process will be blocked until the lock is released; otherwise,
* there's no effect. This allows an isolationtester script to reliably
- * test behaviors where some specified action happens in another backend
- * between parsing and execution of any desired query.
+ * test behaviors where some specified action happens in another backend in
+ * a couple of cases: 1) between parsing and execution of any desired query
+ * when using the planner_hook, 2) between RevalidateCachedQuery() and
+ * ExecutorStart() when using the ExecutorStart_hook.
*
* Copyright (c) 2020-2023, PostgreSQL Global Development Group
*
@@ -22,6 +26,7 @@
#include <limits.h>
+#include "executor/executor.h"
#include "optimizer/planner.h"
#include "utils/builtins.h"
#include "utils/guc.h"
@@ -32,9 +37,11 @@ PG_MODULE_MAGIC;
/* GUC: advisory lock ID to use. Zero disables the feature. */
static int post_planning_lock_id = 0;
+static int executor_start_lock_id = 0;
-/* Save previous planner hook user to be a good citizen */
+/* Save previous hook users to be a good citizen */
static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL;
/* planner_hook function to provide the desired delay */
@@ -70,11 +77,45 @@ delay_execution_planner(Query *parse, const char *query_string,
return result;
}
+/* ExecutorStart_hook function to provide the desired delay */
+static bool
+delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ bool plan_valid;
+
+ /* If enabled, delay by taking and releasing the specified lock */
+ if (executor_start_lock_id != 0)
+ {
+ DirectFunctionCall1(pg_advisory_lock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+ DirectFunctionCall1(pg_advisory_unlock_int8,
+ Int64GetDatum((int64) executor_start_lock_id));
+
+ /*
+ * Ensure that we notice any pending invalidations, since the advisory
+ * lock functions don't do this.
+ */
+ AcceptInvalidationMessages();
+ }
+
+ /* Now start the executor, possibly via a previous hook user */
+ if (prev_ExecutorStart_hook)
+ plan_valid = prev_ExecutorStart_hook(queryDesc, eflags);
+ else
+ plan_valid = standard_ExecutorStart(queryDesc, eflags);
+
+ if (executor_start_lock_id != 0)
+ elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s",
+ plan_valid ? "valid" : "not valid");
+
+ return plan_valid;
+}
+
/* Module load function */
void
_PG_init(void)
{
- /* Set up the GUC to control which lock is used */
+ /* Set up GUCs to control which lock is used */
DefineCustomIntVariable("delay_execution.post_planning_lock_id",
"Sets the advisory lock ID to be locked/unlocked after planning.",
"Zero disables the delay.",
@@ -86,10 +127,22 @@ _PG_init(void)
NULL,
NULL,
NULL);
-
+ DefineCustomIntVariable("delay_execution.executor_start_lock_id",
+ "Sets the advisory lock ID to be locked/unlocked before starting execution.",
+ "Zero disables the delay.",
+ &executor_start_lock_id,
+ 0,
+ 0, INT_MAX,
+ PGC_USERSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
MarkGUCPrefixReserved("delay_execution");
- /* Install our hook */
+ /* Install our hooks. */
prev_planner_hook = planner_hook;
planner_hook = delay_execution_planner;
+ prev_ExecutorStart_hook = ExecutorStart_hook;
+ ExecutorStart_hook = delay_execution_ExecutorStart;
}
diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out
new file mode 100644
index 0000000000..0ac6a17c2b
--- /dev/null
+++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out
@@ -0,0 +1,156 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1prep s2lock s1exec s2dropi s2unlock
+step s1prep: SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1);
+QUERY PLAN
+--------------------------------------------
+Append
+ Subplans Removed: 1
+ -> Bitmap Heap Scan on foo11 foo_1
+ Recheck Cond: (a = $1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = $1)
+(6 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+-----------------------------
+Append
+ Subplans Removed: 1
+ -> Seq Scan on foo11 foo_1
+ Filter: (a = $1)
+(4 rows)
+
+
+starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock
+step s1prep2: SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+--------------------------------------
+Bitmap Heap Scan on foo11 foo
+ Recheck Cond: (a = 1)
+ -> Bitmap Index Scan on foo11_a_idx
+ Index Cond: (a = 1)
+(4 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec2: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec2: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------
+Seq Scan on foo11 foo
+ Filter: (a = 1)
+(2 rows)
+
+
+starting permutation: s1prep3 s2lock s1exec3 s2dropi s2unlock
+step s1prep3: SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3;
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+----------------------------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Index Only Scan using foo11_a_idx on foo11 t1
+ -> Materialize
+ -> Index Scan using foo11_a_idx on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(18 rows)
+
+step s2lock: SELECT pg_advisory_lock(12345);
+pg_advisory_lock
+----------------
+
+(1 row)
+
+step s1exec3: LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; <waiting ...>
+step s2dropi: DROP INDEX foo11_a;
+step s2unlock: SELECT pg_advisory_unlock(12345);
+pg_advisory_unlock
+------------------
+t
+(1 row)
+
+step s1exec3: <... completed>
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid
+s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid
+QUERY PLAN
+---------------------------------------------
+Append
+ -> GroupAggregate
+ Group Key: t1.a
+ -> Merge Join
+ Merge Cond: (t1.a = t2.a)
+ -> Sort
+ Sort Key: t1.a
+ -> Seq Scan on foo11 t1
+ -> Sort
+ Sort Key: t2.a
+ -> Seq Scan on foo11 t2
+ -> GroupAggregate
+ Group Key: t1_1.a
+ -> Merge Join
+ Merge Cond: (t1_1.a = t2_1.a)
+ -> Sort
+ Sort Key: t1_1.a
+ -> Seq Scan on foo2 t1_1
+ -> Sort
+ Sort Key: t2_1.a
+ -> Seq Scan on foo2 t2_1
+(21 rows)
+
diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
new file mode 100644
index 0000000000..3c92cbd5c6
--- /dev/null
+++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec
@@ -0,0 +1,61 @@
+# Test to check that invalidation of cached generic plans during ExecutorStart
+# correctly triggers replanning and re-execution.
+
+setup
+{
+ CREATE TABLE foo (a int, b text) PARTITION BY LIST(a);
+ CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a);
+ CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1);
+ CREATE INDEX foo11_a ON foo1 (a);
+ CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2);
+ CREATE VIEW foov AS SELECT * FROM foo;
+}
+
+teardown
+{
+ DROP VIEW foov;
+ DROP TABLE foo;
+}
+
+session "s1"
+# Append with run-time pruning
+step "s1prep" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q AS SELECT * FROM foov WHERE a = $1;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+
+# no Append case (only one partition selected by the planner)
+step "s1prep2" { SET plan_cache_mode = force_generic_plan;
+ PREPARE q2 AS SELECT * FROM foov WHERE a = 1;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+
+# Append with partition-wise join aggregate and join plans as child subplans
+step "s1prep3" { SET plan_cache_mode = force_generic_plan;
+ SET enable_partitionwise_aggregate = on;
+ SET enable_partitionwise_join = on;
+ PREPARE q3 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+# Executes a generic plan
+step "s1exec" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q (1); }
+step "s1exec2" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q2; }
+step "s1exec3" { LOAD 'delay_execution';
+ SET delay_execution.executor_start_lock_id = 12345;
+ EXPLAIN (COSTS OFF) EXECUTE q3; }
+
+session "s2"
+step "s2lock" { SELECT pg_advisory_lock(12345); }
+step "s2unlock" { SELECT pg_advisory_unlock(12345); }
+step "s2dropi" { DROP INDEX foo11_a; }
+
+# While "s1exec", etc. wait to acquire the advisory lock, "s2drop" is able to
+# drop the index being used in the cached plan. When "s1exec" is then
+# unblocked and initializes the cached plan for execution, it detects the
+# concurrent index drop and causes the cached plan to be discarded and
+# recreated without the index.
+permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock"
+permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock"
+permutation "s1prep3" "s2lock" "s1exec3" "s2dropi" "s2unlock"
--
2.35.3
[application/octet-stream] v44-0001-Make-PlanState-tree-cleanup-non-recursive.patch (28.2K, 7-v44-0001-Make-PlanState-tree-cleanup-non-recursive.patch)
download | inline diff:
From a55bd363690bc4c28047e4b874ce80384e37c49d Mon Sep 17 00:00:00 2001
From: Amit Langote <[email protected]>
Date: Tue, 1 Aug 2023 11:36:24 +0900
Subject: [PATCH v44 1/6] Make PlanState tree cleanup non-recursive
With this change, node type specific subroutines of ExecEndNode()
are no longer required to also clean up the child nodes of a given
node, only its own stuff. Instead, ExecEndPlan() calls
ExecInitNode() directly for each node in the PlanState tree by
iterating over a list (EState.es_planstate_nodes) of all those nodes
built during the ExecInitNode() traversal of the tree.
This changes the order in which the nodes get cleaned up, because
they are now cleaned up in the order in which they are added into
the list which is from leaf-level up to the root, whereas with the
current recursive approach cleanup occurs from the root to the
leaves. The change seems harmless though, because there isn't
necessarily any coupling between of the cleanup actions of parent
and child nodes.
The main motivation behind this change is to allow the cases in
the future where ExecInitNode() traversal of the plan tree may
be aborted in the middle resulting in a partially initialized
PlanState tree. Dealing with that case by making the cleanup
phase walk over a list of successfully initialized nodes seems
better / more robust than making the individual ExecEndNode()
subroutines deal with partially valid PlanState nodes.
---
src/backend/executor/README | 4 +-
src/backend/executor/execMain.c | 36 +++++++-------
src/backend/executor/execProcnode.c | 56 ++++++++++------------
src/backend/executor/execUtils.c | 2 +
src/backend/executor/nodeAgg.c | 4 +-
src/backend/executor/nodeAppend.c | 20 +-------
src/backend/executor/nodeBitmapAnd.c | 23 +--------
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeBitmapOr.c | 23 +--------
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeGather.c | 2 +-
src/backend/executor/nodeGatherMerge.c | 2 +-
src/backend/executor/nodeGroup.c | 5 +-
src/backend/executor/nodeHash.c | 8 +---
src/backend/executor/nodeHashjoin.c | 6 +--
src/backend/executor/nodeIncrementalSort.c | 5 +-
src/backend/executor/nodeLimit.c | 2 +-
src/backend/executor/nodeLockRows.c | 2 +-
src/backend/executor/nodeMaterial.c | 5 +-
src/backend/executor/nodeMemoize.c | 5 +-
src/backend/executor/nodeMergeAppend.c | 20 +-------
src/backend/executor/nodeMergejoin.c | 6 +--
src/backend/executor/nodeModifyTable.c | 7 +--
src/backend/executor/nodeNestloop.c | 6 +--
src/backend/executor/nodeProjectSet.c | 5 +-
src/backend/executor/nodeRecursiveunion.c | 6 +--
src/backend/executor/nodeResult.c | 5 +-
src/backend/executor/nodeSetOp.c | 2 +-
src/backend/executor/nodeSort.c | 5 +-
src/backend/executor/nodeSubqueryscan.c | 5 +-
src/backend/executor/nodeUnique.c | 2 +-
src/backend/executor/nodeWindowAgg.c | 4 +-
src/include/nodes/execnodes.h | 2 +
33 files changed, 80 insertions(+), 214 deletions(-)
diff --git a/src/backend/executor/README b/src/backend/executor/README
index 17775a49e2..67a5c1769b 100644
--- a/src/backend/executor/README
+++ b/src/backend/executor/README
@@ -310,13 +310,13 @@ This is a sketch of control flow for full query processing:
AfterTriggerEndQuery
ExecutorEnd
- ExecEndNode --- recursively releases resources
+ ExecEndPlan --- releases plan resources
FreeExecutorState
frees per-query context and child contexts
FreeQueryDesc
-Per above comments, it's not really critical for ExecEndNode to free any
+Per above comments, it's not really critical for ExecEndPlan to free any
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
be careful to close relations, drop buffer pins, etc, so we do need to scan
the plan state tree to find these sorts of resources.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 4c5a7bbf62..235bb52ccc 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -82,7 +82,7 @@ ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL;
static void InitPlan(QueryDesc *queryDesc, int eflags);
static void CheckValidRowMarkRel(Relation rel, RowMarkType markType);
static void ExecPostprocessPlan(EState *estate);
-static void ExecEndPlan(PlanState *planstate, EState *estate);
+static void ExecEndPlan(EState *estate);
static void ExecutePlan(EState *estate, PlanState *planstate,
bool use_parallel_mode,
CmdType operation,
@@ -500,7 +500,7 @@ standard_ExecutorEnd(QueryDesc *queryDesc)
*/
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndPlan(queryDesc->planstate, estate);
+ ExecEndPlan(estate);
/* do away with our snapshots */
UnregisterSnapshot(estate->es_snapshot);
@@ -1499,23 +1499,21 @@ ExecPostprocessPlan(EState *estate)
* ----------------------------------------------------------------
*/
static void
-ExecEndPlan(PlanState *planstate, EState *estate)
+ExecEndPlan(EState *estate)
{
ListCell *l;
/*
- * shut down the node-type-specific query processing
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (es_plannedstmt->planTree) and those in subplans
+ * (es_plannedstmt->subplans).
*/
- ExecEndNode(planstate);
-
- /*
- * for subplans too
- */
- foreach(l, estate->es_subplanstates)
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *pstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(pstate);
}
/*
@@ -3030,13 +3028,17 @@ EvalPlanQualEnd(EPQState *epqstate)
oldcontext = MemoryContextSwitchTo(estate->es_query_cxt);
- ExecEndNode(epqstate->recheckplanstate);
-
- foreach(l, estate->es_subplanstates)
+ /*
+ * Shut down the node-type-specific query processing for all nodes that
+ * were initialized in InitPlan(). That includes the nodes in both the
+ * main plan tree (epqstate->plan) and those in subplans
+ * (es_plannedstmt->subplans).
+ */
+ foreach(l, estate->es_planstate_nodes)
{
- PlanState *subplanstate = (PlanState *) lfirst(l);
+ PlanState *planstate = (PlanState *) lfirst(l);
- ExecEndNode(subplanstate);
+ ExecEndNode(planstate);
}
/* throw away the per-estate tuple table, some node may have used it */
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index 4d288bc8d4..653f74cf58 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -1,11 +1,13 @@
/*-------------------------------------------------------------------------
*
* execProcnode.c
- * contains dispatch functions which call the appropriate "initialize",
- * "get a tuple", and "cleanup" routines for the given node type.
- * If the node has children, then it will presumably call ExecInitNode,
- * ExecProcNode, or ExecEndNode on its subnodes and do the appropriate
- * processing.
+ * Contains dispatch functions ExecInitNode(), ExecProcNode(), and
+ * ExecEndNode(), which call the appropriate "initialize", "get a tuple",
+ * and "cleanup" routines, respectively, for the given node type.
+ *
+ * While the first two process the node's children recursively, ExecEndNode()
+ * is only concerned with the cleaning of the node itself while the children
+ * are processed by the caller.
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@@ -49,7 +51,9 @@
* Eventually this calls ExecInitNode() on the right and left subplans
* and so forth until the entire plan is initialized. The result
* of ExecInitNode() is a plan state tree built with the same structure
- * as the underlying plan tree.
+ * as the underlying plan tree. (The plan state nodes are also added to
+ * a list in the same order in which they are created for the final
+ * cleanup processing.)
*
* * Then when ExecutorRun() is called, it calls ExecutePlan() which calls
* ExecProcNode() repeatedly on the top node of the plan state tree.
@@ -61,14 +65,10 @@
* form the tuples it returns.
*
* * Eventually ExecSeqScan() stops returning tuples and the nest
- * loop join ends. Lastly, ExecutorEnd() calls ExecEndNode() which
- * calls ExecEndNestLoop() which in turn calls ExecEndNode() on
- * its subplans which result in ExecEndSeqScan().
+ * loop join ends. Lastly, ExecutorEnd() calls ExecEndPlan(), which
+ * in turn calls ExecEndNode() on all the nodes that were initialized:
+ * the two Seq Scans and the Nest Loop in this case.
*
- * This should show how the executor works by having
- * ExecInitNode(), ExecProcNode() and ExecEndNode() dispatch
- * their work to the appropriate node support routines which may
- * in turn call these routines themselves on their subplans.
*/
#include "postgres.h"
@@ -136,6 +136,9 @@ static bool ExecShutdownNode_walker(PlanState *node, void *context);
* 'eflags' is a bitwise OR of flag bits described in executor.h
*
* Returns a PlanState node corresponding to the given Plan node.
+ *
+ * As a side-effect, all PlanState nodes that are created are appended to
+ * estate->es_planstate_nodes for the cleanup processing in ExecEndPlan().
* ------------------------------------------------------------------------
*/
PlanState *
@@ -411,6 +414,10 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
result->instrument = InstrAlloc(1, estate->es_instrument,
result->async_capable);
+ /* And remember for the cleanup processing in ExecEndPlan(). */
+ estate->es_planstate_nodes = lappend(estate->es_planstate_nodes,
+ result);
+
return result;
}
@@ -545,29 +552,18 @@ MultiExecProcNode(PlanState *node)
/* ----------------------------------------------------------------
* ExecEndNode
*
- * Recursively cleans up all the nodes in the plan rooted
- * at 'node'.
+ * Cleans up node
*
- * After this operation, the query plan will not be able to be
- * processed any further. This should be called only after
- * the query plan has been fully executed.
+ * Unlike ExecInitNode(), this does not recurse into child nodes, because
+ * they are processed separately. So the ExecEnd* routine for any given
+ * node type is only responsible for cleaning up its own resources.
* ----------------------------------------------------------------
*/
void
ExecEndNode(PlanState *node)
{
- /*
- * do nothing when we get to the end of a leaf on tree.
- */
- if (node == NULL)
- return;
-
- /*
- * Make sure there's enough stack available. Need to check here, in
- * addition to ExecProcNode() (via ExecProcNodeFirst()), because it's not
- * guaranteed that ExecProcNode() is reached for all nodes.
- */
- check_stack_depth();
+ /* We only ever get called on nodes that were actually initialized. */
+ Assert(node != NULL);
if (node->chgParam != NULL)
{
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index c06b228858..b567165003 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -154,6 +154,8 @@ CreateExecutorState(void)
estate->es_exprcontexts = NIL;
+ estate->es_planstate_nodes = NIL;
+
estate->es_subplanstates = NIL;
estate->es_auxmodifytables = NIL;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 468db94fe5..e9d9ab6bdd 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -4304,7 +4304,6 @@ GetAggInitVal(Datum textInitVal, Oid transtype)
void
ExecEndAgg(AggState *node)
{
- PlanState *outerPlan;
int transno;
int numGroupingSets = Max(node->maxsets, 1);
int setno;
@@ -4367,8 +4366,7 @@ ExecEndAgg(AggState *node)
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closely separately. */
}
void
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..9148d7d3b1 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -376,30 +376,12 @@ ExecAppend(PlanState *pstate)
/* ----------------------------------------------------------------
* ExecEndAppend
- *
- * Shuts down the subscans of the append node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndAppend(AppendState *node)
{
- PlanState **appendplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- appendplans = node->appendplans;
- nplans = node->as_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(appendplans[i]);
+ /* Nothing to do as the nodes in appendplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c
index 4c5eb2b23b..147592f7e2 100644
--- a/src/backend/executor/nodeBitmapAnd.c
+++ b/src/backend/executor/nodeBitmapAnd.c
@@ -168,33 +168,12 @@ MultiExecBitmapAnd(BitmapAndState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapAnd
- *
- * Shuts down the subscans of the BitmapAnd node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapAnd(BitmapAndState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f35df0b8bf..d58ee4f4e1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -667,10 +667,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/*
* release bitmaps and buffers if any
diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c
index 0bf8af9652..736852a0ae 100644
--- a/src/backend/executor/nodeBitmapOr.c
+++ b/src/backend/executor/nodeBitmapOr.c
@@ -186,33 +186,12 @@ MultiExecBitmapOr(BitmapOrState *node)
/* ----------------------------------------------------------------
* ExecEndBitmapOr
- *
- * Shuts down the subscans of the BitmapOr node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndBitmapOr(BitmapOrState *node)
{
- PlanState **bitmapplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- bitmapplans = node->bitmapplans;
- nplans = node->nplans;
-
- /*
- * shut down each of the subscans (that we've initialized)
- */
- for (i = 0; i < nplans; i++)
- {
- if (bitmapplans[i])
- ExecEndNode(bitmapplans[i]);
- }
+ /* Nothing to do as the nodes in bitmapplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index c2139acca0..e6616dd718 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -309,9 +309,7 @@ ExecEndForeignScan(ForeignScanState *node)
else
node->fdwroutine->EndForeignScan(node);
- /* Shut down any outer plan. */
- if (outerPlanState(node))
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
/* Free the exprcontext */
ExecFreeExprContext(&node->ss.ps);
diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c
index 307fc10eea..f7a69f185b 100644
--- a/src/backend/executor/nodeGather.c
+++ b/src/backend/executor/nodeGather.c
@@ -248,7 +248,7 @@ ExecGather(PlanState *pstate)
void
ExecEndGather(GatherState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGather(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c
index 9d5e1a46e9..d357ff0c47 100644
--- a/src/backend/executor/nodeGatherMerge.c
+++ b/src/backend/executor/nodeGatherMerge.c
@@ -288,7 +288,7 @@ ExecGatherMerge(PlanState *pstate)
void
ExecEndGatherMerge(GatherMergeState *node)
{
- ExecEndNode(outerPlanState(node)); /* let children clean up first */
+ /* outerPlan is closed separately. */
ExecShutdownGatherMerge(node);
ExecFreeExprContext(&node->ps);
if (node->ps.ps_ResultTupleSlot)
diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c
index 25a1618952..2badcc7e60 100644
--- a/src/backend/executor/nodeGroup.c
+++ b/src/backend/executor/nodeGroup.c
@@ -226,15 +226,12 @@ ExecInitGroup(Group *node, EState *estate, int eflags)
void
ExecEndGroup(GroupState *node)
{
- PlanState *outerPlan;
-
ExecFreeExprContext(&node->ss.ps);
/* clean up tuple table */
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index 8b5c35b82b..edd2324384 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -413,18 +413,12 @@ ExecInitHash(Hash *node, EState *estate, int eflags)
void
ExecEndHash(HashState *node)
{
- PlanState *outerPlan;
-
/*
* free exprcontext
*/
ExecFreeExprContext(&node->ps);
- /*
- * shut down the subplan
- */
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index 980746128b..8078d7f229 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -879,11 +879,7 @@ ExecEndHashJoin(HashJoinState *node)
ExecClearTuple(node->hj_OuterTupleSlot);
ExecClearTuple(node->hj_HashTupleSlot);
- /*
- * clean up subtrees
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/*
diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 7683e3341c..52b146cfb8 100644
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1101,10 +1101,7 @@ ExecEndIncrementalSort(IncrementalSortState *node)
node->prefixsort_state = NULL;
}
- /*
- * Shut down the subplan.
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO_printf("ExecEndIncrementalSort: sort node shutdown\n");
}
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 425fbfc405..a75099dd73 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -535,7 +535,7 @@ void
ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index e459971d32..55de8d3d65 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -386,7 +386,7 @@ ExecEndLockRows(LockRowsState *node)
{
/* We may have shut down EPQ already, but no harm in another call */
EvalPlanQualEnd(&node->lr_epqstate);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c
index 09632678b0..ef04e9a8e7 100644
--- a/src/backend/executor/nodeMaterial.c
+++ b/src/backend/executor/nodeMaterial.c
@@ -251,10 +251,7 @@ ExecEndMaterial(MaterialState *node)
tuplestore_end(node->tuplestorestate);
node->tuplestorestate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c
index 4f04269e26..61578d4b5c 100644
--- a/src/backend/executor/nodeMemoize.c
+++ b/src/backend/executor/nodeMemoize.c
@@ -1100,10 +1100,7 @@ ExecEndMemoize(MemoizeState *node)
*/
ExecFreeExprContext(&node->ss.ps);
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c
index 21b5726e6e..8aa64944c9 100644
--- a/src/backend/executor/nodeMergeAppend.c
+++ b/src/backend/executor/nodeMergeAppend.c
@@ -310,30 +310,12 @@ heap_compare_slots(Datum a, Datum b, void *arg)
/* ----------------------------------------------------------------
* ExecEndMergeAppend
- *
- * Shuts down the subscans of the MergeAppend node.
- *
- * Returns nothing of interest.
* ----------------------------------------------------------------
*/
void
ExecEndMergeAppend(MergeAppendState *node)
{
- PlanState **mergeplans;
- int nplans;
- int i;
-
- /*
- * get information from the node
- */
- mergeplans = node->mergeplans;
- nplans = node->ms_nplans;
-
- /*
- * shut down each of the subscans
- */
- for (i = 0; i < nplans; i++)
- ExecEndNode(mergeplans[i]);
+ /* Nothing to do as the nodes in mergeplans are closed separately. */
}
void
diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c
index 00f96d045e..7b530d9088 100644
--- a/src/backend/executor/nodeMergejoin.c
+++ b/src/backend/executor/nodeMergejoin.c
@@ -1654,11 +1654,7 @@ ExecEndMergeJoin(MergeJoinState *node)
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
ExecClearTuple(node->mj_MarkedTupleSlot);
- /*
- * shut down the subplans
- */
- ExecEndNode(innerPlanState(node));
- ExecEndNode(outerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
MJ1_printf("ExecEndMergeJoin: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2a5fec8d01..bdbaa4753b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -4397,7 +4397,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* ----------------------------------------------------------------
* ExecEndModifyTable
*
- * Shuts down the plan.
+ * Releases ModifyTable resources.
*
* Returns nothing of interest.
* ----------------------------------------------------------------
@@ -4461,10 +4461,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
EvalPlanQualEnd(&node->mt_epqstate);
- /*
- * shut down subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c
index b3d52e69ec..5cfb50a366 100644
--- a/src/backend/executor/nodeNestloop.c
+++ b/src/backend/executor/nodeNestloop.c
@@ -374,11 +374,7 @@ ExecEndNestLoop(NestLoopState *node)
*/
ExecClearTuple(node->js.ps.ps_ResultTupleSlot);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
NL1_printf("ExecEndNestLoop: %s\n",
"node processing ended");
diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c
index f6ff3dc44c..4a388220ee 100644
--- a/src/backend/executor/nodeProjectSet.c
+++ b/src/backend/executor/nodeProjectSet.c
@@ -330,10 +330,7 @@ ExecEndProjectSet(ProjectSetState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c
index e781003934..aee31c7139 100644
--- a/src/backend/executor/nodeRecursiveunion.c
+++ b/src/backend/executor/nodeRecursiveunion.c
@@ -281,11 +281,7 @@ ExecEndRecursiveUnion(RecursiveUnionState *node)
if (node->tableContext)
MemoryContextDelete(node->tableContext);
- /*
- * close down subplans
- */
- ExecEndNode(outerPlanState(node));
- ExecEndNode(innerPlanState(node));
+ /* outerPlan and innerPlan are closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 4219712d30..a100b144be 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -250,10 +250,7 @@ ExecEndResult(ResultState *node)
*/
ExecClearTuple(node->ps.ps_ResultTupleSlot);
- /*
- * shut down subplans
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
void
diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c
index 4bc2406b89..f7db9a3415 100644
--- a/src/backend/executor/nodeSetOp.c
+++ b/src/backend/executor/nodeSetOp.c
@@ -590,7 +590,7 @@ ExecEndSetOp(SetOpState *node)
MemoryContextDelete(node->tableContext);
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index c6c72c6e67..078d041c40 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -317,10 +317,7 @@ ExecEndSort(SortState *node)
tuplesort_end((Tuplesortstate *) node->tuplesortstate);
node->tuplesortstate = NULL;
- /*
- * shut down the subplan
- */
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
SO1_printf("ExecEndSort: %s\n",
"sort node shutdown");
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 42471bfc04..bc55a82fc3 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -179,10 +179,7 @@ ExecEndSubqueryScan(SubqueryScanState *node)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
- /*
- * close down subquery
- */
- ExecEndNode(node->subplan);
+ /* subplan is closed separately. */
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c
index 45035d74fa..50babacdc8 100644
--- a/src/backend/executor/nodeUnique.c
+++ b/src/backend/executor/nodeUnique.c
@@ -173,7 +173,7 @@ ExecEndUnique(UniqueState *node)
ExecFreeExprContext(&node->ps);
- ExecEndNode(outerPlanState(node));
+ /* outerPlan is closed separately. */
}
diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c
index 310ac23e3a..648cdadc32 100644
--- a/src/backend/executor/nodeWindowAgg.c
+++ b/src/backend/executor/nodeWindowAgg.c
@@ -2681,7 +2681,6 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags)
void
ExecEndWindowAgg(WindowAggState *node)
{
- PlanState *outerPlan;
int i;
release_partition(node);
@@ -2714,8 +2713,7 @@ ExecEndWindowAgg(WindowAggState *node)
pfree(node->perfunc);
pfree(node->peragg);
- outerPlan = outerPlanState(node);
- ExecEndNode(outerPlan);
+ /* outerPlan is closed separately. */
}
/* -----------------
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cb714f4a19..233fb6b4f9 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -671,6 +671,8 @@ typedef struct EState
List *es_exprcontexts; /* List of ExprContexts within EState */
+ List *es_planstate_nodes; /* "flat" list of PlanState nodes */
+
List *es_subplanstates; /* List of PlanState for SubPlans */
List *es_auxmodifytables; /* List of secondary ModifyTableStates */
--
2.35.3
view thread (31+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: generic plans and "initial" pruning
In-Reply-To: <CA+HiwqE=qxN5C-oN5vguBZOZGyDRAMV2EW1pO_hObcpf6X5QwQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox