public inbox for [email protected]help / color / mirror / Atom feed
Re: generic plans and "initial" pruning 22+ messages / 4 participants [nested] [flat]
* Re: generic plans and "initial" pruning @ 2023-01-19 19:39 Tom Lane <[email protected]> 0 siblings, 2 replies; 22+ messages in thread From: Tom Lane @ 2023-01-19 19:39 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers I spent some time re-reading this whole thread, and the more I read the less happy I got. We are adding a lot of complexity and introducing coding hazards that will surely bite somebody someday. And after awhile I had what felt like an epiphany: the whole problem arises because the system is wrongly factored. We should get rid of AcquireExecutorLocks altogether, allowing the plancache to hand back a generic plan that it's not certain of the validity of, and instead integrate the responsibility for acquiring locks into executor startup. It'd have to be optional there, since we don't need new locks in the case of executing a just-planned plan; but we can easily add another eflags bit (EXEC_FLAG_GET_LOCKS or so). Then there has to be a convention whereby the ExecInitNode traversal can return an indicator that "we failed because the plan is stale, please make a new plan". There are a couple reasons why this feels like a good idea: * There's no need for worry about keeping the locking decisions in sync with what executor startup does. * We don't need to add the overhead proposed in the current patch to pass forward data about what got locked/pruned. While that overhead is hopefully less expensive than the locks it saved acquiring, it's still overhead (and in some cases the patch will fail to save acquiring any locks, making it certainly a net negative). * In a successfully built execution state tree, there will simply not be any nodes corresponding to pruned-away, never-locked subplans. As long as code like EXPLAIN follows the state tree and doesn't poke into plan nodes that have no matching state, it's secure against the sort of problems that Robert worried about upthread. While I've not attempted to write any code for this, I can also think of a few issues that'd have to be resolved: * We'd be pushing the responsibility for looping back and re-planning out to fairly high-level calling code. There are only half a dozen callers of GetCachedPlan, so there's not that many places to be touched; but in some of those places the subsequent executor-start call is not close by, so that the necessary refactoring might be pretty painful. I doubt there's anything insurmountable, but we'd definitely be changing some fundamental APIs. * In some cases (views, at least) we need to acquire lock on relations that aren't directly reflected anywhere in the plan tree. So there'd have to be a separate mechanism for getting those locks and rechecking validity afterward. A list of relevant relation OIDs might be enough for that. * We currently do ExecCheckPermissions() before initializing the plan state tree. It won't do to check permissions on relations we haven't yet locked, so that responsibility would have to be moved. Maybe that could also be integrated into the initialization recursion? Not sure. * In the existing usage of AcquireExecutorLocks, if we do decide that the plan is stale then we are able to release all the locks we got before we go off and replan. I'm not certain if that behavior needs to be preserved, but if it does then that would require some additional bookkeeping in the executor. * This approach is optimizing on the assumption that we usually won't need to replan, because if we do then we might waste a fair amount of executor startup overhead before discovering we have to throw all that state away. I think that's clearly the right way to bet, but perhaps somebody else has a different view. Thoughts? regards, tom lane ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-20 03:13 Amit Langote <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-01-20 03:13 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Fri, Jan 20, 2023 at 4:39 AM Tom Lane <[email protected]> wrote: > I spent some time re-reading this whole thread, and the more I read > the less happy I got. Thanks a lot for your time on this. > We are adding a lot of complexity and introducing > coding hazards that will surely bite somebody someday. And after awhile > I had what felt like an epiphany: the whole problem arises because the > system is wrongly factored. We should get rid of AcquireExecutorLocks > altogether, allowing the plancache to hand back a generic plan that > it's not certain of the validity of, and instead integrate the > responsibility for acquiring locks into executor startup. It'd have > to be optional there, since we don't need new locks in the case of > executing a just-planned plan; but we can easily add another eflags > bit (EXEC_FLAG_GET_LOCKS or so). Then there has to be a convention > whereby the ExecInitNode traversal can return an indicator that > "we failed because the plan is stale, please make a new plan". Interesting. The current implementation relies on PlanCacheRelCallback() marking a generic CachedPlan as invalid, so perhaps there will have to be some sharing of state between the plancache and the executor for this to work? > There are a couple reasons why this feels like a good idea: > > * There's no need for worry about keeping the locking decisions in sync > with what executor startup does. > > * We don't need to add the overhead proposed in the current patch to > pass forward data about what got locked/pruned. While that overhead > is hopefully less expensive than the locks it saved acquiring, it's > still overhead (and in some cases the patch will fail to save acquiring > any locks, making it certainly a net negative). > > * In a successfully built execution state tree, there will simply > not be any nodes corresponding to pruned-away, never-locked subplans. > As long as code like EXPLAIN follows the state tree and doesn't poke > into plan nodes that have no matching state, it's secure against the > sort of problems that Robert worried about upthread. I think this is true with the patch as proposed too, but I was still a bit worried about what an ExecutorStart_hook may be doing with an uninitialized plan tree. Maybe we're mandating that the hook must call standard_ExecutorStart() and only work with the finished PlanState tree? > While I've not attempted to write any code for this, I can also > think of a few issues that'd have to be resolved: > > * We'd be pushing the responsibility for looping back and re-planning > out to fairly high-level calling code. There are only half a dozen > callers of GetCachedPlan, so there's not that many places to be > touched; but in some of those places the subsequent executor-start call > is not close by, so that the necessary refactoring might be pretty > painful. I doubt there's anything insurmountable, but we'd definitely > be changing some fundamental APIs. Yeah. I suppose mostly the same place that the current patch is touching to pass around the PartitionPruneResult nodes. > * In some cases (views, at least) we need to acquire lock on relations > that aren't directly reflected anywhere in the plan tree. So there'd > have to be a separate mechanism for getting those locks and rechecking > validity afterward. A list of relevant relation OIDs might be enough > for that. Hmm, a list of only the OIDs wouldn't preserve the lock mode, so maybe a list or bitmapset of the RTIs, something along the lines of PlannedStmt.minLockRelids in the patch? It perhaps even makes sense to make a special list in PlannedStmt for only the views? > * We currently do ExecCheckPermissions() before initializing the > plan state tree. It won't do to check permissions on relations we > haven't yet locked, so that responsibility would have to be moved. > Maybe that could also be integrated into the initialization recursion? > Not sure. Ah, I remember mentioning moving that into ExecGetRangeTableRelation() [1], but I guess that misses relations that are not referenced in the plan tree, such as views. Though maybe that's not a problem if we track views separately as mentioned above. > * In the existing usage of AcquireExecutorLocks, if we do decide > that the plan is stale then we are able to release all the locks > we got before we go off and replan. I'm not certain if that behavior > needs to be preserved, but if it does then that would require some > additional bookkeeping in the executor. I think maybe we'll want to continue to release the existing locks, because if we don't, it's possible we may keep some locks uselessly if replanning might lock a different set of relations. > * This approach is optimizing on the assumption that we usually > won't need to replan, because if we do then we might waste a fair > amount of executor startup overhead before discovering we have > to throw all that state away. I think that's clearly the right > way to bet, but perhaps somebody else has a different view. Not sure if you'd like, because it would still keep the PartitionPruneResult business, but this will be less of a problem if we do the initial pruning at the beginning of InitPlan(), followed by locking, before doing anything else. We would have initialized the QueryDesc and the EState, but only minimally. That also keeps the PartitionPruneResult business local to the executor. Would you like me to hack up a PoC or are you already on that? -- Thanks, Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/CA%2BHiwqG7ZruBmmih3wPsBZ4s0H2EhywrnXEduckY5Hr3fWzPWA%40mail.g... ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-20 03:31 Tom Lane <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Tom Lane @ 2023-01-20 03:31 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers Amit Langote <[email protected]> writes: > On Fri, Jan 20, 2023 at 4:39 AM Tom Lane <[email protected]> wrote: >> I had what felt like an epiphany: the whole problem arises because the >> system is wrongly factored. We should get rid of AcquireExecutorLocks >> altogether, allowing the plancache to hand back a generic plan that >> it's not certain of the validity of, and instead integrate the >> responsibility for acquiring locks into executor startup. > Interesting. The current implementation relies on > PlanCacheRelCallback() marking a generic CachedPlan as invalid, so > perhaps there will have to be some sharing of state between the > plancache and the executor for this to work? Yeah. Thinking a little harder, I think this would have to involve passing a CachedPlan pointer to the executor, and what the executor would do after acquiring each lock is to ask the plancache "hey, do you still think this CachedPlan entry is valid?". In the case where there's a problem, the AcceptInvalidationMessages call involved in lock acquisition would lead to a cache inval that clears the validity flag on the CachedPlan entry, and this would provide an inexpensive way to check if that happened. It might be possible to incorporate this pointer into PlannedStmt instead of passing it separately. >> * In a successfully built execution state tree, there will simply >> not be any nodes corresponding to pruned-away, never-locked subplans. > I think this is true with the patch as proposed too, but I was still a > bit worried about what an ExecutorStart_hook may be doing with an > uninitialized plan tree. Maybe we're mandating that the hook must > call standard_ExecutorStart() and only work with the finished > PlanState tree? It would certainly be incumbent on any such hook to not touch not-yet-locked parts of the plan tree. I'm not particularly concerned about that sort of requirements change, because we'd be breaking APIs all through this area in any case. >> * In some cases (views, at least) we need to acquire lock on relations >> that aren't directly reflected anywhere in the plan tree. So there'd >> have to be a separate mechanism for getting those locks and rechecking >> validity afterward. A list of relevant relation OIDs might be enough >> for that. > Hmm, a list of only the OIDs wouldn't preserve the lock mode, Good point. I wonder if we could integrate this with the RTEPermissionInfo data structure? > Would you like me to hack up a PoC or are you already on that? I'm not planning to work on this myself, I was hoping you would. regards, tom lane ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-20 03:52 Amit Langote <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 2 replies; 22+ messages in thread From: Amit Langote @ 2023-01-20 03:52 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Fri, Jan 20, 2023 at 12:31 PM Tom Lane <[email protected]> wrote: > Amit Langote <[email protected]> writes: > > On Fri, Jan 20, 2023 at 4:39 AM Tom Lane <[email protected]> wrote: > >> I had what felt like an epiphany: the whole problem arises because the > >> system is wrongly factored. We should get rid of AcquireExecutorLocks > >> altogether, allowing the plancache to hand back a generic plan that > >> it's not certain of the validity of, and instead integrate the > >> responsibility for acquiring locks into executor startup. > > > Interesting. The current implementation relies on > > PlanCacheRelCallback() marking a generic CachedPlan as invalid, so > > perhaps there will have to be some sharing of state between the > > plancache and the executor for this to work? > > Yeah. Thinking a little harder, I think this would have to involve > passing a CachedPlan pointer to the executor, and what the executor > would do after acquiring each lock is to ask the plancache "hey, do > you still think this CachedPlan entry is valid?". In the case where > there's a problem, the AcceptInvalidationMessages call involved in > lock acquisition would lead to a cache inval that clears the validity > flag on the CachedPlan entry, and this would provide an inexpensive > way to check if that happened. OK, thanks, this is useful. > It might be possible to incorporate this pointer into PlannedStmt > instead of passing it separately. Yeah, that would be less churn. Though, I wonder if you still hold that PlannedStmt should not be scribbled upon outside the planner as you said upthread [1]? > >> * In a successfully built execution state tree, there will simply > >> not be any nodes corresponding to pruned-away, never-locked subplans. > > > I think this is true with the patch as proposed too, but I was still a > > bit worried about what an ExecutorStart_hook may be doing with an > > uninitialized plan tree. Maybe we're mandating that the hook must > > call standard_ExecutorStart() and only work with the finished > > PlanState tree? > > It would certainly be incumbent on any such hook to not touch > not-yet-locked parts of the plan tree. I'm not particularly concerned > about that sort of requirements change, because we'd be breaking APIs > all through this area in any case. OK. Perhaps something that should be documented around ExecutorStart(). > >> * In some cases (views, at least) we need to acquire lock on relations > >> that aren't directly reflected anywhere in the plan tree. So there'd > >> have to be a separate mechanism for getting those locks and rechecking > >> validity afterward. A list of relevant relation OIDs might be enough > >> for that. > > > Hmm, a list of only the OIDs wouldn't preserve the lock mode, > > Good point. I wonder if we could integrate this with the > RTEPermissionInfo data structure? You mean adding a rellockmode field to RTEPermissionInfo? > > Would you like me to hack up a PoC or are you already on that? > > I'm not planning to work on this myself, I was hoping you would. Alright, I'll try to get something out early next week. Thanks for all the pointers. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/922566.1648784745%40sss.pgh.pa.us ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-20 03:58 Tom Lane <[email protected]> parent: Amit Langote <[email protected]> 1 sibling, 1 reply; 22+ messages in thread From: Tom Lane @ 2023-01-20 03:58 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers Amit Langote <[email protected]> writes: > On Fri, Jan 20, 2023 at 12:31 PM Tom Lane <[email protected]> wrote: >> It might be possible to incorporate this pointer into PlannedStmt >> instead of passing it separately. > Yeah, that would be less churn. Though, I wonder if you still hold > that PlannedStmt should not be scribbled upon outside the planner as > you said upthread [1]? Well, the whole point of that rule is that the executor can't modify a plancache entry. If the plancache itself sets a field in such an entry, that doesn't seem problematic from here. But there's other possibilities if that bothers you; QueryDesc could hold the field, for example. Also, I bet we'd want to copy it into EState for the main initialization recursion. regards, tom lane ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-20 07:19 Amit Langote <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Amit Langote @ 2023-01-20 07:19 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Fri, Jan 20, 2023 at 12:58 PM Tom Lane <[email protected]> wrote: > Amit Langote <[email protected]> writes: > > On Fri, Jan 20, 2023 at 12:31 PM Tom Lane <[email protected]> wrote: > >> It might be possible to incorporate this pointer into PlannedStmt > >> instead of passing it separately. > > > Yeah, that would be less churn. Though, I wonder if you still hold > > that PlannedStmt should not be scribbled upon outside the planner as > > you said upthread [1]? > > Well, the whole point of that rule is that the executor can't modify > a plancache entry. If the plancache itself sets a field in such an > entry, that doesn't seem problematic from here. > > But there's other possibilities if that bothers you; QueryDesc > could hold the field, for example. Also, I bet we'd want to copy > it into EState for the main initialization recursion. QueryDesc sounds good to me, and yes, also a copy in EState in any case. So I started looking at the call sites of CreateQueryDesc() and stopped to look at ExecParallelGetQueryDesc(). AFAICS, we wouldn't need to pass the CachedPlan to a parallel worker's rerun of InitPlan(), because 1) it doesn't make sense to call the plancache in a parallel worker, 2) the leader should already have taken all the locks in necessary for executing a given plan subnode that it intends to pass to a worker in ExecInitGather(). Does that make sense? -- Thanks, Amit Langote EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-01-27 07:01 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 1 sibling, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-01-27 07:01 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Fri, Jan 20, 2023 at 12:52 PM Amit Langote <[email protected]> wrote: > Alright, I'll try to get something out early next week. Thanks for > all the pointers. Sorry for the delay. Attached is what I've come up with so far. I didn't actually go with calling the plancache on every lock taken on a relation, that is, in ExecGetRangeTableRelation(). One thing about doing it that way that I didn't quite like (or didn't see a clean enough way to code) is the need to complicate the ExecInitNode() traversal for handling the abrupt suspension of the ongoing setup of the PlanState tree. So, I decided to keep the current model of locking all the relations that need to be locked before doing anything else in InitPlan(), much as how AcquireExecutorLocks() does it. A new function called from the top of InitPlan that I've called ExecLockRelationsIfNeeded() does that locking after performing the initial pruning in the same manner as the earlier patch did. That does mean that I needed to keep all the adjustments of the pruning code that are required for such out-of-ExecInitNode() invocation of initial pruning, including those PartitionPruneResult to carry the result of that pruning for ExecInitNode()-time reuse, though they no longer need be passed through many unrelated interfaces. Anyways, here's a description of the patches: 0001 adjusts various call sites of ExecutorStart() to cope with the possibility of being asked to recreate a CachedPlan, if one is involved. The main objective here is to have as little stuff as sensible happen between GetCachedPlan() that returned the CachedPlan and ExecutorStart() so as to minimize the chances of missing cleaning up resources that must not be missed. 0002 is preparatory refactoring to make out-of-ExecInitNode() invocation of pruning possible. 0003 moves the responsibility of CachedPlan validation locking into ExecutorStart() as described above. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v31-0001-Move-ExecutorStart-closer-to-GetCachedPlan.patch (47.5K, 2-v31-0001-Move-ExecutorStart-closer-to-GetCachedPlan.patch) download | inline diff: From 4cfc3fdfb4c31a163fc3b0657be77927314cc1ca Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v31 1/3] Move ExecutorStart() closer to GetCachedPlan() This is in preparation for moving CachedPlan validation locking into ExecutorStart(). The intent is to not have many steps between GetCachedPlan() and ExecutorStart() so that if the latter invalidates a CachedPlan, there's not much resource cleanup to worry about. --- contrib/auto_explain/auto_explain.c | 9 +- .../pg_stat_statements/pg_stat_statements.c | 8 +- src/backend/commands/copyto.c | 5 +- src/backend/commands/createas.c | 4 +- src/backend/commands/explain.c | 148 ++++++--- src/backend/commands/extension.c | 3 +- src/backend/commands/matview.c | 4 +- src/backend/commands/portalcmds.c | 2 +- src/backend/commands/prepare.c | 85 +++-- src/backend/executor/execMain.c | 18 +- src/backend/executor/execParallel.c | 9 +- src/backend/executor/functions.c | 3 +- src/backend/executor/spi.c | 47 ++- src/backend/nodes/Makefile | 1 + src/backend/nodes/gen_node_support.pl | 2 + src/backend/tcop/postgres.c | 12 +- src/backend/tcop/pquery.c | 292 +++++++++--------- src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 8 +- src/include/executor/execdesc.h | 4 + src/include/executor/executor.h | 6 +- src/include/nodes/meson.build | 1 + src/include/tcop/pquery.h | 3 +- src/include/utils/portal.h | 2 + 24 files changed, 411 insertions(+), 271 deletions(-) diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c index c3ac27ae99..0f20b97781 100644 --- a/contrib/auto_explain/auto_explain.c +++ b/contrib/auto_explain/auto_explain.c @@ -78,7 +78,8 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL; static ExecutorFinish_hook_type prev_ExecutorFinish = NULL; static ExecutorEnd_hook_type prev_ExecutorEnd = NULL; -static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags); +static void explain_ExecutorStart(QueryDesc *queryDesc, int eflags, + bool *replan); static void explain_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count, bool execute_once); @@ -259,7 +260,7 @@ _PG_init(void) * ExecutorStart hook: start up logging if needed */ static void -explain_ExecutorStart(QueryDesc *queryDesc, int eflags) +explain_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan) { /* * At the beginning of each top-level statement, decide whether we'll @@ -296,9 +297,9 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags) } if (prev_ExecutorStart) - prev_ExecutorStart(queryDesc, eflags); + prev_ExecutorStart(queryDesc, eflags, replan); else - standard_ExecutorStart(queryDesc, eflags); + standard_ExecutorStart(queryDesc, eflags, replan); if (auto_explain_enabled()) { diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index ad1fe44496..76348419ae 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c @@ -325,7 +325,7 @@ static PlannedStmt *pgss_planner(Query *parse, const char *query_string, int cursorOptions, ParamListInfo boundParams); -static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags); +static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan); static void pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count, bool execute_once); @@ -962,12 +962,12 @@ pgss_planner(Query *parse, * ExecutorStart hook: start up tracking if needed */ static void -pgss_ExecutorStart(QueryDesc *queryDesc, int eflags) +pgss_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan) { if (prev_ExecutorStart) - prev_ExecutorStart(queryDesc, eflags); + prev_ExecutorStart(queryDesc, eflags, replan); else - standard_ExecutorStart(queryDesc, eflags); + standard_ExecutorStart(queryDesc, eflags, replan); /* * If query has queryId zero, don't track it. This prevents double diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index 8043b4e9b1..b6d8fa59d5 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -568,7 +569,7 @@ BeginCopyTo(ParseState *pstate, * * ExecutorStart computes a result tupdesc for us */ - ExecutorStart(cstate->queryDesc, 0); + ExecutorStart(cstate->queryDesc, 0, NULL); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..ee33f02602 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,12 +325,12 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, GetIntoRelEFlags(into)); + ExecutorStart(queryDesc, GetIntoRelEFlags(into), NULL); /* run the plan to completion */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 5212a64b1e..fcb227533c 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,94 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv, NULL); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); + + /* One pushed by ExplainQueryDesc(). */ + PopActiveSnapshot(); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * On return, *replan is set to true if cplan is found to have been + * invalidated since its creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv, + bool *replan) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + if (replan) + *replan = false; + ExecutorStart(queryDesc, eflags, replan); + if (replan && *replan) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,30 +598,18 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; + PlannedStmt *plannedstmt = queryDesc->plannedstmt; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; Assert(plannedstmt->commandType != CMD_UTILITY); - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; - /* * We always collect timing for the entire statement, even when node-level * timing is off, so we don't look at es->timing here. (We could skip @@ -546,38 +617,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -658,8 +697,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, FreeQueryDesc(queryDesc); - PopActiveSnapshot(); - /* We need a CCI just in case query expanded to multiple plans */ if (es->analyze) CommandCounterIncrement(); @@ -4854,6 +4891,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index b1509cc505..1493b99beb 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -780,11 +780,12 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); - ExecutorStart(qdesc, 0); + ExecutorStart(qdesc, 0, NULL); ExecutorRun(qdesc, ForwardScanDirection, 0, true); ExecutorFinish(qdesc); ExecutorEnd(qdesc); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..e13b344ba3 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,12 +409,12 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, 0); + ExecutorStart(queryDesc, 0, NULL); /* run the plan */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..9fd27bf07a 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -143,7 +143,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa /* * Start execution, inserting parameters if any. */ - PortalStart(portal, params, 0, GetActiveSnapshot()); + PortalStart(portal, params, 0, GetActiveSnapshot(), NULL); Assert(portal->strategy == PORTAL_ONE_SELECT); diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..c1fa1b72be 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -155,6 +155,7 @@ ExecuteQuery(ParseState *pstate, PreparedStatement *entry; CachedPlan *cplan; List *plan_list; + bool replan; ParamListInfo paramLI = NULL; EState *estate = NULL; Portal portal; @@ -193,6 +194,7 @@ ExecuteQuery(ParseState *pstate, entry->plansource->query_string); /* Replan if needed, and increment plan refcount for portal */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, NULL, NULL); plan_list = cplan->stmt_list; @@ -251,9 +253,16 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, + * it must be recreated if *replan is set. */ - PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + PortalStart(portal, paramLI, eflags, GetActiveSnapshot(), &replan); + + if (replan) + { + MarkPortalFailed(portal); + goto replan; + } (void) PortalRun(portal, count, false, true, dest, dest, qc); @@ -574,7 +583,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -583,6 +592,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, instr_time planduration; BufferUsage bufusage_start, bufusage; + bool replan = true; if (es->buffers) bufusage_start = pgBufferUsage; @@ -618,38 +628,57 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ - cplan = GetCachedPlan(entry->plansource, paramLI, - CurrentResourceOwner, queryEnv); + while (replan) + { + cplan = GetCachedPlan(entry->plansource, paramLI, + CurrentResourceOwner, queryEnv); - INSTR_TIME_SET_CURRENT(planduration); - INSTR_TIME_SUBTRACT(planduration, planstart); + INSTR_TIME_SET_CURRENT(planduration); + INSTR_TIME_SUBTRACT(planduration, planstart); - /* calc differences of buffer counters. */ - if (es->buffers) - { - memset(&bufusage, 0, sizeof(BufferUsage)); - BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); - } + /* calc differences of buffer counters. */ + if (es->buffers) + { + memset(&bufusage, 0, sizeof(BufferUsage)); + BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); + } - plan_list = cplan->stmt_list; + plan_list = cplan->stmt_list; - /* Explain each query */ - foreach(p, plan_list) - { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); + /* Explain each query */ + foreach(p, plan_list) + { + PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); - if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); - else - ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, - paramLI, queryEnv); + if (pstmt->commandType != CMD_UTILITY) + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv, + &replan); + if (replan) + { + ExplainResetOutput(es); + break; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + + /* One pushed by ExplainQueryDesc(). */ + PopActiveSnapshot(); + } + else + ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, + paramLI, queryEnv); - /* No need for CommandCounterIncrement, as ExplainOnePlan did it */ + /* No need for CommandCounterIncrement, as ExplainOnePlan did it */ - /* Separate plans with an appropriate separator */ - if (lnext(plan_list, p) != NULL) - ExplainSeparatePlans(es); + /* Separate plans with an appropriate separator */ + if (lnext(plan_list, p) != NULL) + ExplainSeparatePlans(es); + } } if (estate) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index a5115b9c1f..45c999bcdb 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -119,6 +119,11 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * * eflags contains flag bits as described in executor.h. * + * replan must be non-NULL when executing a cached query plan. On return, + * *replan is set if queryDesc->cplan is found to have been invalidated. In + * that case, callers must recreate the CachedPlan before retrying the + * execution. + * * NB: the CurrentMemoryContext when this is called will become the parent * of the per-query context used for this Executor invocation. * @@ -129,8 +134,10 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * ---------------------------------------------------------------- */ void -ExecutorStart(QueryDesc *queryDesc, int eflags) +ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan) { + Assert(replan != NULL || queryDesc->cplan == NULL); + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -140,13 +147,13 @@ ExecutorStart(QueryDesc *queryDesc, int eflags) pgstat_report_query_id(queryDesc->plannedstmt->queryId, false); if (ExecutorStart_hook) - (*ExecutorStart_hook) (queryDesc, eflags); + (*ExecutorStart_hook) (queryDesc, eflags, replan); else - standard_ExecutorStart(queryDesc, eflags); + standard_ExecutorStart(queryDesc, eflags, replan); } void -standard_ExecutorStart(QueryDesc *queryDesc, int eflags) +standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan) { EState *estate; MemoryContext oldcontext; @@ -2797,7 +2804,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..5f97f5353f 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); @@ -1431,7 +1436,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc) /* Start up the executor */ queryDesc->plannedstmt->jitFlags = fpes->jit_flags; - ExecutorStart(queryDesc, fpes->eflags); + ExecutorStart(queryDesc, fpes->eflags, NULL); /* Special executor initialization steps for parallel workers */ queryDesc->planstate->state->es_query_dsa = area; diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..df37bfb4ed 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, @@ -867,7 +868,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) eflags = EXEC_FLAG_SKIP_TRIGGERS; else eflags = 0; /* default run-to-completion flags */ - ExecutorStart(es->qd, eflags); + ExecutorStart(es->qd, eflags, NULL); } es->status = F_EXEC_RUN; diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index 61f03e3999..9a3398b591 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1578,6 +1578,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, CachedPlanSource *plansource; CachedPlan *cplan; List *stmt_list; + bool replan; char *query_string; Snapshot snapshot; MemoryContext oldcontext; @@ -1657,6 +1658,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, */ /* Replan if needed, and increment plan refcount for portal */ +replan: cplan = GetCachedPlan(plansource, paramLI, NULL, _SPI_current->queryEnv); stmt_list = cplan->stmt_list; @@ -1766,9 +1768,16 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if *replan is set. */ - PortalStart(portal, paramLI, 0, snapshot); + PortalStart(portal, paramLI, 0, snapshot, &replan); + + if (replan) + { + MarkPortalFailed(portal); + goto replan; + } Assert(portal->strategy != PORTAL_MULTI_QUERY); @@ -2548,6 +2557,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2657,6 +2667,8 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; + bool replan = false; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2664,14 +2676,28 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + ExecutorStart(qdesc, eflags, &replan); + if (replan) + { + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2846,10 +2872,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2893,14 +2918,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile index af12c64878..7fb0d2d202 100644 --- a/src/backend/nodes/Makefile +++ b/src/backend/nodes/Makefile @@ -52,6 +52,7 @@ node_headers = \ access/tsmapi.h \ commands/event_trigger.h \ commands/trigger.h \ + executor/execdesc.h \ executor/tuptable.h \ foreign/fdwapi.h \ nodes/bitmapset.h \ diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl index b3c1ead496..74f83f12a6 100644 --- a/src/backend/nodes/gen_node_support.pl +++ b/src/backend/nodes/gen_node_support.pl @@ -63,6 +63,7 @@ my @all_input_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/bitmapset.h @@ -87,6 +88,7 @@ my @nodetag_only_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/lockoptions.h diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..1617b93ecc 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1195,7 +1195,7 @@ exec_simple_query(const char *query_string) /* * Start the portal. No parameters here. */ - PortalStart(portal, NULL, 0, InvalidSnapshot); + PortalStart(portal, NULL, 0, InvalidSnapshot, NULL); /* * Select the appropriate output format: text unless we are doing a @@ -1597,6 +1597,7 @@ exec_bind_message(StringInfo input_message) int16 *rformats = NULL; CachedPlanSource *psrc; CachedPlan *cplan; + bool replan; Portal portal; char *query_string; char *saved_stmt_name; @@ -1971,6 +1972,7 @@ exec_bind_message(StringInfo input_message) * will be generated in MessageContext. The plan refcount will be * assigned to the Portal, so it will be released at portal destruction. */ +replan: cplan = GetCachedPlan(psrc, params, NULL, NULL); /* @@ -1993,7 +1995,13 @@ exec_bind_message(StringInfo input_message) /* * And we're ready to start portal execution. */ - PortalStart(portal, params, 0, InvalidSnapshot); + PortalStart(portal, params, 0, InvalidSnapshot, &replan); + + if (replan) + { + MarkPortalFailed(portal); + goto replan; + } /* * Apply the result format requests to the portal. diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..97de5c53e3 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -75,8 +71,10 @@ CreateQueryDesc(PlannedStmt *plannedstmt, { QueryDesc *qd = (QueryDesc *) palloc(sizeof(QueryDesc)); + qd->type = T_QueryDesc; qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +114,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,15 +345,16 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless *replan is set to true, in which case, + * the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, - int eflags, Snapshot snapshot) + int eflags, Snapshot snapshot, + bool *replan) { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -443,20 +362,21 @@ PortalStart(Portal portal, ParamListInfo params, Assert(PortalIsValid(portal)); Assert(portal->status == PORTAL_DEFINED); + if (replan) + *replan = false; + /* * Set up global portal context pointers. */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +392,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +415,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +424,48 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ - ExecutorStart(queryDesc, myeflags); + ExecutorStart(queryDesc, myeflags, replan); + if (replan && *replan) + { + Assert(queryDesc->cplan); + PopActiveSnapshot(); + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -536,29 +477,6 @@ PortalStart(Portal portal, ParamListInfo params, PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -581,7 +499,61 @@ PortalStart(Portal portal, ParamListInfo params, break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool pushed_active_snapshot = false; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* Must set snapshot before starting executor. */ + if (!pushed_active_snapshot && !is_utility) + { + PushActiveSnapshot(GetTransactionSnapshot()); + pushed_active_snapshot = true; + } + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + pushed_active_snapshot ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalMultiRun() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + if (!is_utility) + { + ExecutorStart(queryDesc, 0, replan); + if (replan && *replan) + { + Assert(queryDesc->cplan); + if (pushed_active_snapshot) + PopActiveSnapshot(); + goto early_exit; + } + } + } + + if (pushed_active_snapshot) + PopActiveSnapshot(); + } + portal->tupDesc = NULL; break; } @@ -594,19 +566,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1164,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1185,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = lfirst_node(QueryDesc, qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1241,7 +1213,7 @@ PortalRunMulti(Portal portal, */ if (!active_snapshot_set) { - Snapshot snapshot = GetTransactionSnapshot(); + Snapshot snapshot = qdesc->snapshot; /* If told to, register the snapshot and save in portal */ if (setHoldSnapshot) @@ -1271,23 +1243,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1333,19 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + /* portal->queryDesc is free'd by PortalCleanup(). */ + if (qdesc != portal->queryDesc) + { + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); + } } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..ea35adfb3d 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,12 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv, + bool *replan); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..4b7368a0dc 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -57,6 +60,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index e7e25c057e..63f3d09804 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -62,7 +62,7 @@ /* Hook for plugins to get control in ExecutorStart() */ -typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags); +typedef void (*ExecutorStart_hook_type) (QueryDesc *queryDesc, int eflags, bool *replan); extern PGDLLIMPORT ExecutorStart_hook_type ExecutorStart_hook; /* Hook for plugins to get control in ExecutorRun() */ @@ -187,8 +187,8 @@ ExecGetJunkAttribute(TupleTableSlot *slot, AttrNumber attno, bool *isNull) /* * prototypes from functions in execMain.c */ -extern void ExecutorStart(QueryDesc *queryDesc, int eflags); -extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags); +extern void ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan); +extern void standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan); extern void ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count, bool execute_once); extern void standard_ExecutorRun(QueryDesc *queryDesc, diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build index efe0834afb..a8fdd9e176 100644 --- a/src/include/nodes/meson.build +++ b/src/include/nodes/meson.build @@ -13,6 +13,7 @@ node_support_input_i = [ 'access/tsmapi.h', 'commands/event_trigger.h', 'commands/trigger.h', + 'executor/execdesc.h', 'executor/tuptable.h', 'foreign/fdwapi.h', 'nodes/bitmapset.h', diff --git a/src/include/tcop/pquery.h b/src/include/tcop/pquery.h index a5e65b98aa..08783f1b43 100644 --- a/src/include/tcop/pquery.h +++ b/src/include/tcop/pquery.h @@ -30,7 +30,8 @@ extern List *FetchPortalTargetList(Portal portal); extern List *FetchStatementTargetList(Node *stmt); extern void PortalStart(Portal portal, ParamListInfo params, - int eflags, Snapshot snapshot); + int eflags, Snapshot snapshot, + bool *replan); extern void PortalSetResultFormat(Portal portal, int nFormats, int16 *formats); diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..af059e30f8 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,8 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ -- 2.35.3 [application/octet-stream] v31-0003-Move-CachedPlan-validation-locking-to-ExecutorSt.patch (38.4K, 3-v31-0003-Move-CachedPlan-validation-locking-to-ExecutorSt.patch) download | inline diff: From 0522447f5816211ac3e32ebc6920d7f7805718d6 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Thu, 26 Jan 2023 10:52:24 +0900 Subject: [PATCH v31 3/3] Move CachedPlan validation locking to ExecutorStart() --- src/backend/executor/execMain.c | 163 +++++++++++++++++++++++-- src/backend/executor/execParallel.c | 38 +++++- src/backend/executor/execPartition.c | 90 +++++++++++--- src/backend/executor/execUtils.c | 8 +- src/backend/executor/nodeAppend.c | 11 +- src/backend/executor/nodeMergeAppend.c | 5 +- src/backend/optimizer/plan/setrefs.c | 36 ++++++ src/backend/utils/cache/plancache.c | 146 +++++++--------------- src/include/executor/execPartition.h | 8 +- src/include/executor/execdesc.h | 6 + src/include/executor/executor.h | 2 + src/include/nodes/execnodes.h | 1 + src/include/nodes/pathnodes.h | 4 +- src/include/nodes/plannodes.h | 31 ++++- src/include/utils/plancache.h | 1 + 15 files changed, 404 insertions(+), 146 deletions(-) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 45c999bcdb..68743d5f66 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -49,6 +49,7 @@ #include "commands/matview.h" #include "commands/trigger.h" #include "executor/execdebug.h" +#include "executor/execPartition.h" #include "executor/nodeSubplan.h" #include "foreign/fdwapi.h" #include "jit/jit.h" @@ -64,6 +65,7 @@ #include "utils/lsyscache.h" #include "utils/memutils.h" #include "utils/partcache.h" +#include "utils/plancache.h" #include "utils/rls.h" #include "utils/ruleutils.h" #include "utils/snapmgr.h" @@ -79,7 +81,12 @@ ExecutorEnd_hook_type ExecutorEnd_hook = NULL; ExecutorCheckPerms_hook_type ExecutorCheckPerms_hook = NULL; /* decls for local routines only used within this module */ -static void InitPlan(QueryDesc *queryDesc, int eflags); +static void InitPlan(QueryDesc *queryDesc, int eflags, bool *replan); +static void ExecLockRelationsIfNeeded(QueryDesc *queryDesc, bool *replan); +static Bitmapset *ExecDoInitialPartitionPruning(PlannedStmt *stmt, + EState *estate); +static void AcquireExecutorLocks(Bitmapset *lockRelids, EState *estate, + bool acquire); static void CheckValidRowMarkRel(Relation rel, RowMarkType markType); static void ExecPostprocessPlan(EState *estate); static void ExecEndPlan(PlanState *planstate, EState *estate); @@ -270,7 +277,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags, bool *replan) /* * Initialize the plan state tree */ - InitPlan(queryDesc, eflags); + InitPlan(queryDesc, eflags, replan); MemoryContextSwitchTo(oldcontext); } @@ -801,7 +808,7 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) * ---------------------------------------------------------------- */ static void -InitPlan(QueryDesc *queryDesc, int eflags) +InitPlan(QueryDesc *queryDesc, int eflags, bool *replan) { CmdType operation = queryDesc->operation; PlannedStmt *plannedstmt = queryDesc->plannedstmt; @@ -814,19 +821,26 @@ InitPlan(QueryDesc *queryDesc, int eflags) int i; /* - * Do permissions checks and save the list for later use. - */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); - estate->es_rteperminfos = plannedstmt->permInfos; - - /* - * initialize the node's execution state + * Initialize es_range_table and es_relations. */ ExecInitRangeTable(estate, rangeTable); estate->es_plannedstmt = plannedstmt; estate->es_part_prune_infos = plannedstmt->partPruneInfos; + /* + * Acquire locks on relations referenced in the plan if it comes + * from a CachedPlan after performing "initial" partition pruning. + * Results of pruning, if any, are saved in es_part_prune_results. + */ + ExecLockRelationsIfNeeded(queryDesc, replan); + + /* + * Do permissions checks and save the list for later use. + */ + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + estate->es_rteperminfos = plannedstmt->permInfos; + /* * Next, build the ExecRowMark array from the PlanRowMark(s), if any. */ @@ -982,6 +996,133 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->planstate = planstate; } +/* + * ExecLockRelationsIfNeeded + * Lock relations that a query's plan depends on if the plan comes + * from a CachedPlan + * + * On return, we have all acquired the locks needed to run the plan. + * Also *replan is set to true if acquiring those locks would have invalidated + * the CachedPlan. + */ +static void +ExecLockRelationsIfNeeded(QueryDesc *queryDesc, bool *replan) +{ + PlannedStmt *plannedstmt = queryDesc->plannedstmt; + EState *estate = queryDesc->estate; + CachedPlan *cplan = queryDesc->cplan; + Bitmapset *allLockRelids; + + /* Nothing to do if the plan tree is not cached. */ + if (cplan == NULL || cplan->is_oneshot) + return; + + Assert(plannedstmt); + Assert(replan); + *replan = false; + + /* + * Temporarily signal to ExecGetRangeTableRelation() that it must take + * take a lock. This is needed for CreatePartitionPruneState() to be + * able to open parent partitioned tables using + * ExecGetRangeTableRelation(). + */ + estate->es_top_eflags |= EXEC_FLAG_GET_LOCKS; + + allLockRelids = plannedstmt->minLockRelids; + if (plannedstmt->containsInitialPruning) + { + Bitmapset *partRelids = ExecDoInitialPartitionPruning(plannedstmt, + estate); + + allLockRelids = bms_add_members(allLockRelids, partRelids); + } + + /* Done with it. */ + estate->es_top_eflags &= ~EXEC_FLAG_GET_LOCKS; + + /* Acquire locks. */ + AcquireExecutorLocks(allLockRelids, estate, true); + + /* Check if acquiring those locks has invalidated the plan. */ + *replan = !CachedPlanStillValid(cplan); + + /* Release useless locks if needed. */ + if (*replan) + AcquireExecutorLocks(allLockRelids, estate, false); +} + +/* + * ExecDoInitialPartitionPruning + * Perform initial partition pruning if needed by the plan + * + * The return value is the set of RT indexes of surviving partitions. + * A list of PartitionPruneResult with an element for each in + * plannedstmt->partPruneInfos is saved in estate->es_part_prune_results. + */ +static Bitmapset * +ExecDoInitialPartitionPruning(PlannedStmt *plannedstmt, EState *estate) +{ + ListCell *lc; + Bitmapset *lockPartRelids = NULL; + + Assert(plannedstmt->containsInitialPruning); + Assert(plannedstmt->partPruneInfos); + + foreach(lc, plannedstmt->partPruneInfos) + { + PartitionPruneInfo *pruneinfo = lfirst_node(PartitionPruneInfo, lc); + PartitionPruneState *prunestate; + PartitionPruneResult *pruneresult; + Bitmapset *validsubplans; + + /* No PlanState here; unnecessary for "initial" pruning. */ + prunestate = ExecCreatePartitionPruneState(NULL, estate, pruneinfo, + true, false); + validsubplans = ExecFindMatchingSubPlans(prunestate, true, + &lockPartRelids); + + pruneresult = makeNode(PartitionPruneResult); + pruneresult->root_parent_relids = bms_copy(pruneinfo->root_parent_relids); + pruneresult->validsubplans = bms_copy(validsubplans); + estate->es_part_prune_results = lappend(estate->es_part_prune_results, + pruneresult); + } + + return lockPartRelids; +} + +/* + * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; + * or release them if acquire is false. + */ +static void +AcquireExecutorLocks(Bitmapset *lockRelids, EState *estate, bool acquire) +{ + int rti; + + rti = -1; + while ((rti = bms_next_member(lockRelids, rti)) > 0) + { + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + if (!(rte->rtekind == RTE_RELATION || + (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) + continue; + + /* + * Acquire the appropriate type of lock on each relation OID. Note + * that we don't actually try to open the rel, and hence will not + * fail if it's been dropped entirely --- we'll just transiently + * acquire a non-conflicting lock. + */ + if (acquire) + LockRelationOid(rte->relid, rte->rellockmode); + else + UnlockRelationOid(rte->relid, rte->rellockmode); + } +} + /* * Check that a proposed result relation is a legal target for the operation * @@ -1396,7 +1537,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked by the planner. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index 1f5d6d4d64..5c967451ce 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -66,6 +66,7 @@ #define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xE000000000000008) #define PARALLEL_KEY_JIT_INSTRUMENTATION UINT64CONST(0xE000000000000009) #define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xE00000000000000A) +#define PARALLEL_KEY_PARTITION_PRUNE_RESULTS UINT64CONST(0xE00000000000000B) #define PARALLEL_TUPLE_QUEUE_SIZE 65536 @@ -599,12 +600,15 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate, FixedParallelExecutorState *fpes; char *pstmt_data; char *pstmt_space; + char *part_prune_results_data; + char *part_prune_results_space; char *paramlistinfo_space; BufferUsage *bufusage_space; WalUsage *walusage_space; SharedExecutorInstrumentation *instrumentation = NULL; SharedJitInstrumentation *jit_instrumentation = NULL; int pstmt_len; + int part_prune_results_len; int paramlistinfo_len; int instrumentation_len = 0; int jit_instrumentation_len = 0; @@ -633,6 +637,7 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate, /* Fix up and serialize plan to be sent to workers. */ pstmt_data = ExecSerializePlan(planstate->plan, estate); + part_prune_results_data = nodeToString(estate->es_part_prune_results); /* Create a parallel context. */ pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers); @@ -659,6 +664,11 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate, shm_toc_estimate_chunk(&pcxt->estimator, pstmt_len); shm_toc_estimate_keys(&pcxt->estimator, 1); + /* Estimate space for serialized List of PartitionPruneResult. */ + part_prune_results_len = strlen(part_prune_results_data) + 1; + shm_toc_estimate_chunk(&pcxt->estimator, part_prune_results_len); + shm_toc_estimate_keys(&pcxt->estimator, 1); + /* Estimate space for serialized ParamListInfo. */ paramlistinfo_len = EstimateParamListSpace(estate->es_param_list_info); shm_toc_estimate_chunk(&pcxt->estimator, paramlistinfo_len); @@ -753,6 +763,12 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate, memcpy(pstmt_space, pstmt_data, pstmt_len); shm_toc_insert(pcxt->toc, PARALLEL_KEY_PLANNEDSTMT, pstmt_space); + /* Store serialized List of PartitionPruneResult */ + part_prune_results_space = shm_toc_allocate(pcxt->toc, part_prune_results_len); + memcpy(part_prune_results_space, part_prune_results_data, part_prune_results_len); + shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARTITION_PRUNE_RESULTS, + part_prune_results_space); + /* Store serialized ParamListInfo. */ paramlistinfo_space = shm_toc_allocate(pcxt->toc, paramlistinfo_len); shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARAMLISTINFO, paramlistinfo_space); @@ -1234,8 +1250,11 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, int instrument_options) { char *pstmtspace; + char *part_prune_results_space; char *paramspace; PlannedStmt *pstmt; + QueryDesc *queryDesc; + List *part_prune_results; ParamListInfo paramLI; char *queryString; @@ -1246,6 +1265,11 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, pstmtspace = shm_toc_lookup(toc, PARALLEL_KEY_PLANNEDSTMT, false); pstmt = (PlannedStmt *) stringToNode(pstmtspace); + /* Reconstruct leader-supplied PartitionPruneResult. */ + part_prune_results_space = + shm_toc_lookup(toc, PARALLEL_KEY_PARTITION_PRUNE_RESULTS, false); + part_prune_results = (List *) stringToNode(part_prune_results_space); + /* Reconstruct ParamListInfo. */ paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); @@ -1255,11 +1279,15 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, * here even if the containing plan tree may have come from one in the * leader. */ - return CreateQueryDesc(pstmt, - NULL, - queryString, - GetActiveSnapshot(), InvalidSnapshot, - receiver, paramLI, NULL, instrument_options); + queryDesc = CreateQueryDesc(pstmt, + NULL, + queryString, + GetActiveSnapshot(), InvalidSnapshot, + receiver, paramLI, NULL, instrument_options); + + queryDesc->part_prune_results = part_prune_results; + + return queryDesc; } /* diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 4b91bb7403..09e0d7aa9c 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -196,7 +196,8 @@ static void PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate, static void find_matching_subplans_recurse(PartitionPruningData *prunedata, PartitionedRelPruningData *pprune, bool initial_prune, - Bitmapset **validsubplans); + Bitmapset **validsubplans, + Bitmapset **scan_leafpart_rtis); /* @@ -1782,8 +1783,10 @@ adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap) * * On return, *initially_valid_subplans is assigned the set of indexes of * child subplans that must be initialized along with the parent plan node. - * Initial pruning is performed here if needed and in that case only the - * surviving subplans' indexes are added. + * That set is computed by either performing the "initial pruning" here or + * reusing the one present in EState.es_part_prune_results[part_prune_index] + * if it has been set, which it would be if ExecDoInitialPartitionPruning() + * would have done the initial pruning. * * If subplans are indeed pruned, subplan_map arrays contained in the returned * PartitionPruneState are re-sequenced to not count those, though only if the @@ -1796,9 +1799,10 @@ ExecInitPartitionPruning(PlanState *planstate, Bitmapset *root_parent_relids, Bitmapset **initially_valid_subplans) { - PartitionPruneState *prunestate; + PartitionPruneState *prunestate = NULL; EState *estate = planstate->state; PartitionPruneInfo *pruneinfo; + PartitionPruneResult *pruneresult = NULL; /* Obtain the pruneinfo we need, and make sure it's the right one */ pruneinfo = list_nth(estate->es_part_prune_infos, part_prune_index); @@ -1814,22 +1818,56 @@ ExecInitPartitionPruning(PlanState *planstate, /* We may need an expression context to evaluate partition exprs */ ExecAssignExprContext(estate, planstate); - /* Create the working data structure for pruning */ - prunestate = ExecCreatePartitionPruneState(planstate, estate, pruneinfo, - pruneinfo->needs_init_pruning, - pruneinfo->needs_exec_pruning); + /* Initial pruning already done if es_part_prune_results has been set. */ + if (estate->es_part_prune_results) + { + pruneresult = list_nth_node(PartitionPruneResult, + estate->es_part_prune_results, + part_prune_index); + if (!bms_equal(root_parent_relids, pruneinfo->root_parent_relids)) + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("mismatching PartitionPruneInfo and PartitionPruneResult at part_prune_index %d", + part_prune_index), + errdetail_internal("prunresult relids %s, pruneinfo relids %s", + bmsToString(pruneresult->root_parent_relids), + bmsToString(pruneinfo->root_parent_relids))); + } + + if (pruneresult == NULL || pruneinfo->needs_exec_pruning) + { + /* We may need an expression context to evaluate partition exprs */ + ExecAssignExprContext(estate, planstate); + + /* + * Create the working data structure for pruning. No need to consider + * initial pruning steps if we have a PartitionPruneResult. + */ + prunestate = ExecCreatePartitionPruneState(planstate, estate, + pruneinfo, + pruneresult == NULL, + pruneinfo->needs_exec_pruning); + } /* * Perform an initial partition prune pass, if required. */ - if (prunestate->do_initial_prune) - *initially_valid_subplans = ExecFindMatchingSubPlans(prunestate, true); + if (pruneresult) + { + *initially_valid_subplans = bms_copy(pruneresult->validsubplans); + } + else if (prunestate && prunestate->do_initial_prune) + { + *initially_valid_subplans = ExecFindMatchingSubPlans(prunestate, true, + NULL); + } else { - /* No pruning, so we'll need to initialize all subplans */ + /* No initial pruning, so we'll need to initialize all subplans */ Assert(n_total_subplans > 0); *initially_valid_subplans = bms_add_range(NULL, 0, n_total_subplans - 1); + return prunestate; } /* @@ -1837,7 +1875,8 @@ ExecInitPartitionPruning(PlanState *planstate, * that were removed above due to initial pruning. No need to do this if * no steps were removed. */ - if (bms_num_members(*initially_valid_subplans) < n_total_subplans) + if (prunestate && + bms_num_members(*initially_valid_subplans) < n_total_subplans) { /* * We can safely skip this when !do_exec_prune, even though that @@ -2295,10 +2334,14 @@ PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate, * Pass initial_prune if PARAM_EXEC Params cannot yet be evaluated. This * differentiates the initial executor-time pruning step from later * runtime pruning. + * + * RT indexes of leaf partitions scanned by the chosen subplans are added to + * *scan_leafpart_rtis if the pointer is non-NULL. */ Bitmapset * ExecFindMatchingSubPlans(PartitionPruneState *prunestate, - bool initial_prune) + bool initial_prune, + Bitmapset **scan_leafpart_rtis) { Bitmapset *result = NULL; MemoryContext oldcontext; @@ -2333,10 +2376,10 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate, */ pprune = &prunedata->partrelprunedata[0]; find_matching_subplans_recurse(prunedata, pprune, initial_prune, - &result); + &result, scan_leafpart_rtis); /* Expression eval may have used space in ExprContext too */ - if (pprune->exec_pruning_steps) + if (pprune->exec_pruning_steps && !initial_prune) ResetExprContext(pprune->exec_context.exprcontext); } @@ -2347,6 +2390,8 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate, /* Copy result out of the temp context before we reset it */ result = bms_copy(result); + if (scan_leafpart_rtis) + *scan_leafpart_rtis = bms_copy(*scan_leafpart_rtis); MemoryContextReset(prunestate->prune_context); @@ -2357,13 +2402,15 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate, * find_matching_subplans_recurse * Recursive worker function for ExecFindMatchingSubPlans * - * Adds valid (non-prunable) subplan IDs to *validsubplans + * Adds valid (non-prunable) subplan IDs to *validsubplans and RT indexes of + * of the corresponding leaf partitions to *scan_leafpart_rtis (if asked for). */ static void find_matching_subplans_recurse(PartitionPruningData *prunedata, PartitionedRelPruningData *pprune, bool initial_prune, - Bitmapset **validsubplans) + Bitmapset **validsubplans, + Bitmapset **scan_leafpart_rtis) { Bitmapset *partset; int i; @@ -2390,8 +2437,14 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata, while ((i = bms_next_member(partset, i)) >= 0) { if (pprune->subplan_map[i] >= 0) + { *validsubplans = bms_add_member(*validsubplans, pprune->subplan_map[i]); + Assert(pprune->rti_map[i] > 0); + if (scan_leafpart_rtis) + *scan_leafpart_rtis = bms_add_member(*scan_leafpart_rtis, + pprune->rti_map[i]); + } else { int partidx = pprune->subpart_map[i]; @@ -2399,7 +2452,8 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata, if (partidx >= 0) find_matching_subplans_recurse(prunedata, &prunedata->partrelprunedata[partidx], - initial_prune, validsubplans); + initial_prune, validsubplans, + scan_leafpart_rtis); else { /* diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index c33a3c0bec..035ed8a872 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -140,6 +140,7 @@ CreateExecutorState(void) estate->es_param_exec_vals = NULL; estate->es_queryEnv = NULL; + estate->es_part_prune_results = NIL; estate->es_query_cxt = qcontext; @@ -800,7 +801,12 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + /* + * Must take a lock on the relation if we got here by way of + * ExecLockRelationsIfNeeded(). + */ + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index cb25499b3f..2f585793da 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -156,7 +156,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) * subplan, we can fill as_valid_subplans immediately, preventing * later calls to ExecFindMatchingSubPlans. */ - if (!prunestate->do_exec_prune && nplans > 0) + if (appendstate->as_prune_state == NULL || + (!appendstate->as_prune_state->do_exec_prune && nplans > 0)) appendstate->as_valid_subplans = bms_add_range(NULL, 0, nplans - 1); } else @@ -578,7 +579,7 @@ choose_next_subplan_locally(AppendState *node) } else if (node->as_valid_subplans == NULL) node->as_valid_subplans = - ExecFindMatchingSubPlans(node->as_prune_state, false); + ExecFindMatchingSubPlans(node->as_prune_state, false, NULL); whichplan = -1; } @@ -643,7 +644,7 @@ choose_next_subplan_for_leader(AppendState *node) if (node->as_valid_subplans == NULL) { node->as_valid_subplans = - ExecFindMatchingSubPlans(node->as_prune_state, false); + ExecFindMatchingSubPlans(node->as_prune_state, false, NULL); /* * Mark each invalid plan as finished to allow the loop below to @@ -718,7 +719,7 @@ choose_next_subplan_for_worker(AppendState *node) else if (node->as_valid_subplans == NULL) { node->as_valid_subplans = - ExecFindMatchingSubPlans(node->as_prune_state, false); + ExecFindMatchingSubPlans(node->as_prune_state, false, NULL); mark_invalid_subplans_as_finished(node); } @@ -869,7 +870,7 @@ ExecAppendAsyncBegin(AppendState *node) if (node->as_valid_subplans == NULL) { node->as_valid_subplans = - ExecFindMatchingSubPlans(node->as_prune_state, false); + ExecFindMatchingSubPlans(node->as_prune_state, false, NULL); classify_matching_subplans(node); } diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..c653084515 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -104,7 +104,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) * subplan, we can fill ms_valid_subplans immediately, preventing * later calls to ExecFindMatchingSubPlans. */ - if (!prunestate->do_exec_prune && nplans > 0) + if (mergestate->ms_prune_state == NULL || + (!mergestate->ms_prune_state->do_exec_prune && nplans > 0)) mergestate->ms_valid_subplans = bms_add_range(NULL, 0, nplans - 1); } else @@ -219,7 +220,7 @@ ExecMergeAppend(PlanState *pstate) */ if (node->ms_valid_subplans == NULL) node->ms_valid_subplans = - ExecFindMatchingSubPlans(node->ms_prune_state, false); + ExecFindMatchingSubPlans(node->ms_prune_state, false, NULL); /* * First time through: pull the first tuple from each valid subplan, diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index b4fa8d90bc..ff363be811 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -372,6 +372,7 @@ set_plan_references(PlannerInfo *root, Plan *plan) { PartitionPruneInfo *pruneinfo = lfirst(lc); ListCell *l; + Bitmapset *leafpart_rtis = NULL; pruneinfo->root_parent_relids = offset_relid_set(pruneinfo->root_parent_relids, rtoffset); @@ -383,17 +384,52 @@ set_plan_references(PlannerInfo *root, Plan *plan) foreach(l2, prune_infos) { PartitionedRelPruneInfo *pinfo = lfirst(l2); + int i; /* RT index of the table to which the pinfo belongs. */ pinfo->rtindex += rtoffset; + + /* Also of the leaf partitions that might be scanned. */ + for (i = 0; i < pinfo->nparts; i++) + { + if (pinfo->rti_map[i] > 0 && pinfo->subplan_map[i] >= 0) + { + pinfo->rti_map[i] += rtoffset; + leafpart_rtis = bms_add_member(leafpart_rtis, + pinfo->rti_map[i]); + } + } } } + if (pruneinfo->needs_init_pruning) + { + glob->containsInitialPruning = true; + + /* + * Delete the leaf partition RTIs from the set of relations to be + * locked by AcquireExecutorLocks(). The actual set of leaf + * partitions to be locked is computed by + * ExecLockRelationsIfNeeded(). + */ + glob->minLockRelids = bms_del_members(glob->minLockRelids, + leafpart_rtis); + } + glob->partPruneInfos = lappend(glob->partPruneInfos, pruneinfo); glob->containsInitialPruning |= pruneinfo->needs_init_pruning; } + /* + * It seems worth doing a bms_copy() on glob->minLockRelids if we deleted + * bits from it above to get rid of any empty tail bits. It seems better + * for the loop over this set in AcquireExecutorLocks() to not have to go + * through those useless bit words. + */ + if (glob->containsInitialPruning) + glob->minLockRelids = bms_copy(glob->minLockRelids); + return result; } diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index f113170140..af5e9b1609 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,69 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); + return cplan->is_valid; +} - return false; +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor after it has finished taking locks on a plan tree + * in a CachedPlan. + */ +bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return GenericPlanIsValid(cplan); } /* @@ -1126,9 +1132,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1362,6 +1365,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, /* * Reject if AcquireExecutorLocks would have anything to do. This is * probably unnecessary given the previous check, but let's be safe. + * XXX - maybe remove? */ foreach(lc, plan->stmt_list) { @@ -1735,62 +1739,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - Bitmapset *allLockRelids; - int rti; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - Assert(plannedstmt->minLockRelids == NULL); - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - allLockRelids = plannedstmt->minLockRelids; - rti = -1; - while ((rti = bms_next_member(allLockRelids, rti)) > 0) - { - RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h index 21d85a7809..526f5781da 100644 --- a/src/include/executor/execPartition.h +++ b/src/include/executor/execPartition.h @@ -133,5 +133,11 @@ extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate, bool consider_initial_steps, bool consider_exec_steps); extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate, - bool initial_prune); + bool initial_prune, + Bitmapset **scan_leafpart_rtis); +extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate, + EState *estate, + PartitionPruneInfo *pruneinfo, + bool consider_initial_steps, + bool consider_exec_steps); #endif /* EXECPARTITION_H */ diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index 4b7368a0dc..595297df6c 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -46,6 +46,12 @@ typedef struct QueryDesc QueryEnvironment *queryEnv; /* query environment passed in */ int instrument_options; /* OR of InstrumentOption flags */ + /* + * Used by ExecParallelGetQueryDesc() to save the result of initial + * partition pruning performed by the leader. + */ + List *part_prune_results; /* list of PartitionPruneResult */ + /* These fields are set by ExecutorStart */ TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index 63f3d09804..755e231675 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -59,6 +59,8 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should ExecGetRangeTableRelation() + * lock relations? */ /* Hook for plugins to get control in ExecutorStart() */ diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 20f4c8b35f..b361592e2d 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -620,6 +620,7 @@ typedef struct EState List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ + List *es_part_prune_results; /* QueryDesc.part_prune_results */ const char *es_sourceText; /* Source text from QueryDesc */ JunkFilter *es_junkFilter; /* top-level junk filter, if any */ diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d00b5dcb03..83e5c665c7 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -134,8 +134,8 @@ typedef struct PlannerGlobal bool containsInitialPruning; /* - * Indexes of all range table entries; for AcquireExecutorLocks()'s - * perusal. + * Indexes of all range table entries except those of leaf partitions + * scanned by prunable subplans; for AcquireExecutorLocks() perusal. */ Bitmapset *minLockRelids; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 7b53f990e0..e76e945c8c 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -82,8 +82,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ - Bitmapset *minLockRelids; /* Indexes of all range table entries; for - * AcquireExecutorLocks()'s perusal */ + Bitmapset *minLockRelids; /* Indexes of all range table entries except + * those of leaf partitions scanned by + * prunable subplans */ /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ @@ -1575,6 +1576,32 @@ typedef struct PartitionPruneStepCombine List *source_stepids; } PartitionPruneStepCombine; +/*---------------- + * PartitionPruneResult + * + * The result of performing ExecPartitionDoInitialPruning() on a given + * PartitionPruneInfo. + * + * root_parent_relids is same as PartitionPruneInfo.root_parent_relids. It's + * there for cross-checking in ExecInitPartitionPruning() that the + * PartitionPruneResult and the PartitionPruneInfo at a given index in + * EState.es_part_prune_results and EState.es_part_prune_infos, respectively, + * belong to the same parent plan node. + * + * validsubplans contains the indexes of subplans remaining after performing + * initial pruning by calling ExecFindMatchingSubPlans() on the + * PartitionPruneInfo. + * + * This is used to store the result of initial partition pruning that is + * peformed in ExecDoInitialPartitionPruning(). + */ +typedef struct PartitionPruneResult +{ + NodeTag type; + + Bitmapset *root_parent_relids; + Bitmapset *validsubplans; +} PartitionPruneResult; /* * Plan invalidation info diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..7c664bad35 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,7 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); +extern bool CachedPlanStillValid(CachedPlan *cplan); extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, -- 2.35.3 [application/octet-stream] v31-0002-Preparatory-refactoring-before-reworking-CachedP.patch (21.4K, 4-v31-0002-Preparatory-refactoring-before-reworking-CachedP.patch) download | inline diff: From 936ef111a9b515c0d0111637a22959ea62e92b5d Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Tue, 13 Dec 2022 11:58:07 +0900 Subject: [PATCH v31 2/3] Preparatory refactoring before reworking CachedPlan locking Remember the RT indexes of RTEs that AcquireExecutorLocks() must look at to consider locking in a bitmapset, so that nstead of looping over the range table to find those RTEs, it can look them up using the RT indexes set in the bitmapset. This also adds some extra information related to execution-time pruning to the relevant plan nodes. --- src/backend/executor/execParallel.c | 1 + src/backend/executor/execPartition.c | 34 +++++++++++++++------- src/backend/nodes/readfuncs.c | 8 ++++-- src/backend/optimizer/plan/planner.c | 2 ++ src/backend/optimizer/plan/setrefs.c | 12 ++++++++ src/backend/partitioning/partprune.c | 42 ++++++++++++++++++++++++++-- src/backend/utils/cache/plancache.c | 10 +++++-- src/include/executor/execPartition.h | 6 ++++ src/include/nodes/nodes.h | 1 + src/include/nodes/pathnodes.h | 11 ++++++++ src/include/nodes/plannodes.h | 19 +++++++++++++ 11 files changed, 128 insertions(+), 18 deletions(-) diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index 5f97f5353f..1f5d6d4d64 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -182,6 +182,7 @@ ExecSerializePlan(Plan *plan, EState *estate) pstmt->transientPlan = false; pstmt->dependsOnRole = false; pstmt->parallelModeNeeded = false; + pstmt->containsInitialPruning = false; /* workers need not know! */ pstmt->planTree = plan; pstmt->partPruneInfos = estate->es_part_prune_infos; pstmt->rtable = estate->es_range_table; diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 651ad24fc1..4b91bb7403 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -184,8 +184,6 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel, int maxfieldlen); static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri); static List *adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap); -static PartitionPruneState *CreatePartitionPruneState(PlanState *planstate, - PartitionPruneInfo *pruneinfo); static void InitPartitionPruneContext(PartitionPruneContext *context, List *pruning_steps, PartitionDesc partdesc, @@ -1759,6 +1757,11 @@ adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap) * account for initial pruning possibly having eliminated some of the * subplans. * + * ExecCreatePartitionPruneState: + * A sub-routine of ExecInitPartitionPruning() that creates the + * PartitionPruneState from a given PartitionPruneInfo. Exported for the + * use by callers that don't need to do ExecInitPartitionPruning(). + * * ExecFindMatchingSubPlans: * Returns indexes of matching subplans after evaluating the expressions * that are safe to evaluate at a given point. This function is first @@ -1812,7 +1815,9 @@ ExecInitPartitionPruning(PlanState *planstate, ExecAssignExprContext(estate, planstate); /* Create the working data structure for pruning */ - prunestate = CreatePartitionPruneState(planstate, pruneinfo); + prunestate = ExecCreatePartitionPruneState(planstate, estate, pruneinfo, + pruneinfo->needs_init_pruning, + pruneinfo->needs_exec_pruning); /* * Perform an initial partition prune pass, if required. @@ -1849,7 +1854,7 @@ ExecInitPartitionPruning(PlanState *planstate, } /* - * CreatePartitionPruneState + * ExecCreatePartitionPruneState * Build the data structure required for calling ExecFindMatchingSubPlans * * 'planstate' is the parent plan node's execution state. @@ -1865,15 +1870,18 @@ ExecInitPartitionPruning(PlanState *planstate, * re-evaluate which partitions match the pruning steps provided in each * PartitionedRelPruneInfo. */ -static PartitionPruneState * -CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) +PartitionPruneState * +ExecCreatePartitionPruneState(PlanState *planstate, EState *estate, + PartitionPruneInfo *pruneinfo, + bool consider_initial_steps, + bool consider_exec_steps) { - EState *estate = planstate->state; PartitionPruneState *prunestate; int n_part_hierarchies; ListCell *lc; int i; - ExprContext *econtext = planstate->ps_ExprContext; + ExprContext *econtext = planstate ? planstate->ps_ExprContext : + GetPerTupleExprContext(estate); /* For data reading, executor always omits detached partitions */ if (estate->es_partition_directory == NULL) @@ -1955,6 +1963,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) Assert(partdesc->nparts >= pinfo->nparts); pprune->nparts = partdesc->nparts; pprune->subplan_map = palloc(sizeof(int) * partdesc->nparts); + pprune->rti_map = palloc(sizeof(Index) * partdesc->nparts); if (partdesc->nparts == pinfo->nparts) { /* @@ -1965,6 +1974,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) pprune->subpart_map = pinfo->subpart_map; memcpy(pprune->subplan_map, pinfo->subplan_map, sizeof(int) * pinfo->nparts); + memcpy(pprune->rti_map, pinfo->rti_map, + sizeof(int) * pinfo->nparts); /* * Double-check that the list of unpruned relations has not @@ -2015,6 +2026,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) pinfo->subplan_map[pd_idx]; pprune->subpart_map[pp_idx] = pinfo->subpart_map[pd_idx]; + pprune->rti_map[pp_idx] = + pinfo->rti_map[pd_idx]; pd_idx++; } else @@ -2022,6 +2035,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) /* this partdesc entry is not in the plan */ pprune->subplan_map[pp_idx] = -1; pprune->subpart_map[pp_idx] = -1; + pprune->rti_map[pp_idx] = 0; } } @@ -2043,7 +2057,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * Initialize pruning contexts as needed. */ pprune->initial_pruning_steps = pinfo->initial_pruning_steps; - if (pinfo->initial_pruning_steps) + if (consider_initial_steps && pinfo->initial_pruning_steps) { InitPartitionPruneContext(&pprune->initial_context, pinfo->initial_pruning_steps, @@ -2053,7 +2067,7 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) prunestate->do_initial_prune = true; } pprune->exec_pruning_steps = pinfo->exec_pruning_steps; - if (pinfo->exec_pruning_steps) + if (consider_exec_steps && pinfo->exec_pruning_steps) { InitPartitionPruneContext(&pprune->exec_context, pinfo->exec_pruning_steps, diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index f3629cdfd1..caf2a60493 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -158,6 +158,11 @@ token = pg_strtok(&length); /* skip :fldname */ \ local_node->fldname = readIntCols(len) +/* Read an Index array */ +#define READ_INDEX_ARRAY(fldname, len) \ + token = pg_strtok(&length); /* skip :fldname */ \ + local_node->fldname = readIndexCols(len) + /* Read a bool array */ #define READ_BOOL_ARRAY(fldname, len) \ token = pg_strtok(&length); /* skip :fldname */ \ @@ -800,7 +805,6 @@ fnname(int numCols) \ */ READ_SCALAR_ARRAY(readAttrNumberCols, int16, atoi) READ_SCALAR_ARRAY(readOidCols, Oid, atooid) -/* outfuncs.c has writeIndexCols, but we don't yet need that here */ -/* READ_SCALAR_ARRAY(readIndexCols, Index, atoui) */ +READ_SCALAR_ARRAY(readIndexCols, Index, atoui) READ_SCALAR_ARRAY(readIntCols, int, atoi) READ_SCALAR_ARRAY(readBoolCols, bool, strtobool) diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 05f44faf6e..2b7238cf24 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -525,8 +525,10 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->parallelModeNeeded = glob->parallelModeNeeded; result->planTree = top_plan; result->partPruneInfos = glob->partPruneInfos; + result->containsInitialPruning = glob->containsInitialPruning; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->minLockRelids = glob->minLockRelids; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 85ba9d1ca1..b4fa8d90bc 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -279,6 +279,16 @@ set_plan_references(PlannerInfo *root, Plan *plan) */ add_rtes_to_flat_rtable(root, false); + /* + * Add the query's adjusted range of RT indexes to glob->minLockRelids. + * The adjusted RT indexes of prunable relations will be deleted from the + * set below where PartitionPruneInfos are processed. + */ + glob->minLockRelids = + bms_add_range(glob->minLockRelids, + rtoffset + 1, + rtoffset + list_length(root->parse->rtable)); + /* * Adjust RT indexes of PlanRowMarks and add to final rowmarks list */ @@ -377,9 +387,11 @@ set_plan_references(PlannerInfo *root, Plan *plan) /* RT index of the table to which the pinfo belongs. */ pinfo->rtindex += rtoffset; } + } glob->partPruneInfos = lappend(glob->partPruneInfos, pruneinfo); + glob->containsInitialPruning |= pruneinfo->needs_init_pruning; } return result; diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index 510145e3c0..9ae41053da 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@ -144,7 +144,9 @@ static List *make_partitionedrel_pruneinfo(PlannerInfo *root, List *prunequal, Bitmapset *partrelids, int *relid_subplan_map, - Bitmapset **matchedsubplans); + Bitmapset **matchedsubplans, + bool *needs_init_pruning, + bool *needs_exec_pruning); static void gen_partprune_steps(RelOptInfo *rel, List *clauses, PartClauseTarget target, GeneratePruningStepsContext *context); @@ -234,6 +236,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, int *relid_subplan_map; ListCell *lc; int i; + bool needs_init_pruning = false; + bool needs_exec_pruning = false; /* * Scan the subpaths to see which ones are scans of partition child @@ -313,12 +317,16 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, Bitmapset *partrelids = (Bitmapset *) lfirst(lc); List *pinfolist; Bitmapset *matchedsubplans = NULL; + bool partrel_needs_init_pruning; + bool partrel_needs_exec_pruning; pinfolist = make_partitionedrel_pruneinfo(root, parentrel, prunequal, partrelids, relid_subplan_map, - &matchedsubplans); + &matchedsubplans, + &partrel_needs_init_pruning, + &partrel_needs_exec_pruning); /* When pruning is possible, record the matched subplans */ if (pinfolist != NIL) @@ -327,6 +335,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, allmatchedsubplans = bms_join(matchedsubplans, allmatchedsubplans); } + + needs_init_pruning |= partrel_needs_init_pruning; + needs_exec_pruning |= partrel_needs_exec_pruning; } pfree(relid_subplan_map); @@ -342,6 +353,8 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, pruneinfo = makeNode(PartitionPruneInfo); pruneinfo->root_parent_relids = parentrel->relids; pruneinfo->prune_infos = prunerelinfos; + pruneinfo->needs_init_pruning = needs_init_pruning; + pruneinfo->needs_exec_pruning = needs_exec_pruning; /* * Some subplans may not belong to any of the identified partitioned rels. @@ -442,13 +455,19 @@ add_part_relids(List *allpartrelids, Bitmapset *partrelids) * If we cannot find any useful run-time pruning steps, return NIL. * However, on success, each rel identified in partrelids will have * an element in the result list, even if some of them are useless. + * *needs_init_pruning and *needs_exec_pruning are set to indicate whether + * the pruning steps contained in the returned PartitionedRelPruneInfos + * can be performed during executor startup and during execution, + * respectively. */ static List * make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *prunequal, Bitmapset *partrelids, int *relid_subplan_map, - Bitmapset **matchedsubplans) + Bitmapset **matchedsubplans, + bool *needs_init_pruning, + bool *needs_exec_pruning) { RelOptInfo *targetpart = NULL; List *pinfolist = NIL; @@ -459,6 +478,10 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, int rti; int i; + /* Will find out below. */ + *needs_init_pruning = false; + *needs_exec_pruning = false; + /* * Examine each partitioned rel, constructing a temporary array to map * from planner relids to index of the partitioned rel, and building a @@ -546,6 +569,9 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, * executor per-scan pruning steps. This first pass creates startup * pruning steps and detects whether there's any possibly-useful quals * that would require per-scan pruning. + * + * In the first pass, we note whether the 2nd pass is necessary by + * noting the presence of EXEC parameters. */ gen_partprune_steps(subpart, partprunequal, PARTTARGET_INITIAL, &context); @@ -620,6 +646,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, pinfo->execparamids = execparamids; /* Remaining fields will be filled in the next loop */ + /* record which types of pruning steps we've seen so far */ + if (initial_pruning_steps != NIL) + *needs_init_pruning = true; + if (exec_pruning_steps != NIL) + *needs_exec_pruning = true; + pinfolist = lappend(pinfolist, pinfo); } @@ -647,6 +679,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, int *subplan_map; int *subpart_map; Oid *relid_map; + Index *rti_map; /* * Construct the subplan and subpart maps for this partitioning level. @@ -659,6 +692,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, subpart_map = (int *) palloc(nparts * sizeof(int)); memset(subpart_map, -1, nparts * sizeof(int)); relid_map = (Oid *) palloc0(nparts * sizeof(Oid)); + rti_map = (Index *) palloc0(nparts * sizeof(Index)); present_parts = NULL; i = -1; @@ -673,6 +707,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, subplan_map[i] = subplanidx = relid_subplan_map[partrel->relid] - 1; subpart_map[i] = subpartidx = relid_subpart_map[partrel->relid] - 1; relid_map[i] = planner_rt_fetch(partrel->relid, root)->relid; + rti_map[i] = partrel->relid; if (subplanidx >= 0) { present_parts = bms_add_member(present_parts, i); @@ -697,6 +732,7 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, pinfo->subplan_map = subplan_map; pinfo->subpart_map = subpart_map; pinfo->relid_map = relid_map; + pinfo->rti_map = rti_map; } pfree(relid_subpart_map); diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..f113170140 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -1747,7 +1747,8 @@ AcquireExecutorLocks(List *stmt_list, bool acquire) foreach(lc1, stmt_list) { PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; + Bitmapset *allLockRelids; + int rti; if (plannedstmt->commandType == CMD_UTILITY) { @@ -1760,14 +1761,17 @@ AcquireExecutorLocks(List *stmt_list, bool acquire) */ Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); + Assert(plannedstmt->minLockRelids == NULL); if (query) ScanQueryForLocks(query, acquire); continue; } - foreach(lc2, plannedstmt->rtable) + allLockRelids = plannedstmt->minLockRelids; + rti = -1; + while ((rti = bms_next_member(allLockRelids, rti)) > 0) { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); + RangeTblEntry *rte = rt_fetch(rti, plannedstmt->rtable); if (!(rte->rtekind == RTE_RELATION || (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) diff --git a/src/include/executor/execPartition.h b/src/include/executor/execPartition.h index ee487e42dd..21d85a7809 100644 --- a/src/include/executor/execPartition.h +++ b/src/include/executor/execPartition.h @@ -45,6 +45,7 @@ extern void ExecCleanupTupleRouting(ModifyTableState *mtstate, * nparts Length of subplan_map[] and subpart_map[]. * subplan_map Subplan index by partition index, or -1. * subpart_map Subpart index by partition index, or -1. + * rti_map Range table index by partition index, or 0. * present_parts A Bitmapset of the partition indexes that we * have subplans or subparts for. * initial_pruning_steps List of PartitionPruneSteps used to @@ -61,6 +62,7 @@ typedef struct PartitionedRelPruningData int nparts; int *subplan_map; int *subpart_map; + Index *rti_map; Bitmapset *present_parts; List *initial_pruning_steps; List *exec_pruning_steps; @@ -126,6 +128,10 @@ extern PartitionPruneState *ExecInitPartitionPruning(PlanState *planstate, int part_prune_index, Bitmapset *root_parent_relids, Bitmapset **initially_valid_subplans); +extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate, EState *estate, + PartitionPruneInfo *pruneinfo, + bool consider_initial_steps, + bool consider_exec_steps); extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate, bool initial_prune); #endif /* EXECPARTITION_H */ diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index 10752e8011..1de8f3fadc 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -218,6 +218,7 @@ extern struct Bitmapset *readBitmapset(void); extern uintptr_t readDatum(bool typbyval); extern bool *readBoolCols(int numCols); extern int *readIntCols(int numCols); +extern Index *readIndexCols(int numCols); extern Oid *readOidCols(int numCols); extern int16 *readAttrNumberCols(int numCols); diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index 2d1d8f4bcd..d00b5dcb03 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -128,6 +128,17 @@ typedef struct PlannerGlobal /* List of PartitionPruneInfo contained in the plan */ List *partPruneInfos; + /* + * Do any of those PartitionPruneInfos have initial pruning steps in them? + */ + bool containsInitialPruning; + + /* + * Indexes of all range table entries; for AcquireExecutorLocks()'s + * perusal. + */ + Bitmapset *minLockRelids; + /* OIDs of relations the plan depends on */ List *relationOids; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index c1234fcf36..7b53f990e0 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -73,11 +73,18 @@ typedef struct PlannedStmt List *partPruneInfos; /* List of PartitionPruneInfo contained in the * plan */ + bool containsInitialPruning; /* Do any of those PartitionPruneInfos + * have initial pruning steps in them? + */ + List *rtable; /* list of RangeTblEntry nodes */ List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + Bitmapset *minLockRelids; /* Indexes of all range table entries; for + * AcquireExecutorLocks()'s perusal */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ @@ -1417,6 +1424,13 @@ typedef struct PlanRowMark * prune_infos List of Lists containing PartitionedRelPruneInfo nodes, * one sublist per run-time-prunable partition hierarchy * appearing in the parent plan node's subplans. + * + * needs_init_pruning Does any of the PartitionedRelPruneInfos in + * prune_infos have its initial_pruning_steps set? + * + * needs_exec_pruning Does any of the PartitionedRelPruneInfos in + * prune_infos have its exec_pruning_steps set? + * * other_subplans Indexes of any subplans that are not accounted for * by any of the PartitionedRelPruneInfo nodes in * "prune_infos". These subplans must not be pruned. @@ -1428,6 +1442,8 @@ typedef struct PartitionPruneInfo NodeTag type; Bitmapset *root_parent_relids; List *prune_infos; + bool needs_init_pruning; + bool needs_exec_pruning; Bitmapset *other_subplans; } PartitionPruneInfo; @@ -1472,6 +1488,9 @@ typedef struct PartitionedRelPruneInfo /* relation OID by partition index, or 0 */ Oid *relid_map pg_node_attr(array_size(nparts)); + /* Range table index by partition index, or 0. */ + Index *rti_map pg_node_attr(array_size(nparts)); + /* * initial_pruning_steps shows how to prune during executor startup (i.e., * without use of any PARAM_EXEC Params); it is NIL if no startup pruning -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-02-02 14:49 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-02-02 14:49 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Fri, Jan 27, 2023 at 4:01 PM Amit Langote <[email protected]> wrote: > On Fri, Jan 20, 2023 at 12:52 PM Amit Langote <[email protected]> wrote: > > Alright, I'll try to get something out early next week. Thanks for > > all the pointers. > > Sorry for the delay. Attached is what I've come up with so far. > > I didn't actually go with calling the plancache on every lock taken on > a relation, that is, in ExecGetRangeTableRelation(). One thing about > doing it that way that I didn't quite like (or didn't see a clean > enough way to code) is the need to complicate the ExecInitNode() > traversal for handling the abrupt suspension of the ongoing setup of > the PlanState tree. OK, I gave this one more try and attached is what I came up with. This adds a ExecPlanStillValid(), which is called right after anything that may in turn call ExecGetRangeTableRelation() which has been taught to lock a relation if EXEC_FLAG_GET_LOCKS has been passed in EState.es_top_eflags. That includes all ExecInitNode() calls, and a few other functions that call ExecGetRangeTableRelation() directly, such as ExecOpenScanRelation(). If ExecPlanStillValid() returns false, that is, if EState.es_cachedplan is found to have been invalidated after a lock being taken by ExecGetRangeTableRelation(), whatever funcion called it must return immediately and so must its caller and so on. ExecEndPlan() seems to be able to clean up after a partially finished attempt of initializing a PlanState tree in this way. Maybe my preliminary testing didn't catch cases where pointers to resources that are normally put into the nodes of a PlanState tree are now left dangling, because a partially built PlanState tree is not accessible to ExecEndPlan; QueryDesc.planstate would remain NULL in such cases. Maybe there's only es_tupleTable and es_relations that needs to be explicitly released and the rest is taken care of by resetting the ExecutorState context. On testing, I'm afraid we're going to need something like src/test/modules/delay_execution to test that concurrent changes to relation(s) in PlannedStmt.relationOids that occur somewhere between RevalidateCachedQuery() and InitPlan() result in the latter to be aborted and that it is handled correctly. It seems like it is only the locking of partitions (that are not present in an unplanned Query and thus not protected by AcquirePlannerLocks()) that can trigger replanning of a CachedPlan, so any tests we write should involve partitions. Should this try to test as many plan shapes as possible though given the uncertainty around ExecEndPlan() robustness or should manual auditing suffice to be sure that nothing's broken? On possibly needing to move permission checking to occur *after* taking locks, I realized that we don't really need to, because no relation that needs its permissions should be unlocked by the time we get to ExecCheckPermissions(); note we only check permissions of tables that are present in the original parse tree and RevalidateCachedQuery() should have locked those. I found a couple of exceptions to that invariant in that views sometimes appear not to be in the set of relations that RevalidateCachedQuery() locks. So, I invented PlannedStmt.viewRelations, a list of RT indexes of view RTEs that is populated in setrefs.c. ExecLockViewRelations() called before ExecCheckPermissions() locks those. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v32-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch (87.9K, 2-v32-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From d48cb6fe06f7d3d98adb36299966daff7df25a3b Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v32] Move AcquireExecutorLocks()'s responsibility into the executor --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 142 ++++++---- src/backend/commands/extension.c | 1 + src/backend/commands/matview.c | 2 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 29 +- src/backend/executor/execMain.c | 98 ++++++- src/backend/executor/execParallel.c | 7 +- src/backend/executor/execPartition.c | 4 + src/backend/executor/execProcnode.c | 5 + src/backend/executor/execUtils.c | 5 +- src/backend/executor/functions.c | 1 + src/backend/executor/nodeAgg.c | 2 + src/backend/executor/nodeAppend.c | 4 + src/backend/executor/nodeBitmapAnd.c | 2 + src/backend/executor/nodeBitmapHeapscan.c | 4 + src/backend/executor/nodeBitmapOr.c | 2 + src/backend/executor/nodeCustom.c | 2 + src/backend/executor/nodeForeignscan.c | 4 + src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 2 + src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 4 + src/backend/executor/nodeIncrementalSort.c | 2 + src/backend/executor/nodeIndexonlyscan.c | 2 + src/backend/executor/nodeIndexscan.c | 2 + src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 2 + src/backend/executor/nodeMemoize.c | 2 + src/backend/executor/nodeMergeAppend.c | 4 + src/backend/executor/nodeMergejoin.c | 4 + src/backend/executor/nodeModifyTable.c | 7 + src/backend/executor/nodeNestloop.c | 4 + src/backend/executor/nodeProjectSet.c | 2 + src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 2 + src/backend/executor/nodeSamplescan.c | 2 + src/backend/executor/nodeSeqscan.c | 2 + src/backend/executor/nodeSetOp.c | 2 + src/backend/executor/nodeSort.c | 2 + src/backend/executor/nodeSubqueryscan.c | 2 + src/backend/executor/nodeTidrangescan.c | 2 + src/backend/executor/nodeTidscan.c | 2 + src/backend/executor/nodeUnique.c | 2 + src/backend/executor/nodeWindowAgg.c | 2 + src/backend/executor/spi.c | 44 +++- src/backend/nodes/Makefile | 1 + src/backend/nodes/gen_node_support.pl | 2 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 ++++ src/backend/tcop/postgres.c | 8 + src/backend/tcop/pquery.c | 291 +++++++++++---------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 +++------- src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 12 + src/include/nodes/execnodes.h | 2 + src/include/nodes/meson.build | 1 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 15 ++ src/include/utils/portal.h | 4 + 72 files changed, 698 insertions(+), 332 deletions(-) diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index 8043b4e9b1..a438c547e8 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index fbbf28cf06..8fdc966a73 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,89 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,29 +593,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - - Assert(plannedstmt->commandType != CMD_UTILITY); - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -546,38 +611,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4851,6 +4884,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index b1509cc505..e2f79cc7a7 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -780,6 +780,7 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..17d457ccfb 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,7 +409,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..6c72b46f07 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,17 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, + * it must be recreated if *replan is set. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +582,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +626,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +648,20 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index a5115b9c1f..47bc6a1f3a 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -119,6 +119,11 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * * eflags contains flag bits as described in executor.h. * + * replan must be non-NULL when executing a cached query plan. On return, + * *replan is set if queryDesc->cplan is found to have been invalidated. In + * that case, callers must recreate the CachedPlan before retrying the + * execution. + * * NB: the CurrentMemoryContext when this is called will become the parent * of the per-query context used for this Executor invocation. * @@ -131,6 +136,10 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +591,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +804,43 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } +/* + * Lock view relations in a given query's range table. + */ +static void +ExecLockViewRelations(List *viewRelations, EState *estate, bool acquire) +{ + ListCell *lc; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + if (acquire) + LockRelationOid(rte->relid, rte->rellockmode); + else + UnlockRelationOid(rte->relid, rte->rellockmode); + } +} /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -807,17 +857,21 @@ InitPlan(QueryDesc *queryDesc, int eflags) int i; /* - * Do permissions checks and save the list for later use. + * initialize the node's execution state */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); - estate->es_rteperminfos = plannedstmt->permInfos; + ExecInitRangeTable(estate, rangeTable); + + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, true); /* - * initialize the node's execution state + * Do permissions checks and save the list for later use. */ - ExecInitRangeTable(estate, rangeTable); + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + estate->es_rteperminfos = plannedstmt->permInfos; estate->es_plannedstmt = plannedstmt; + estate->es_cachedplan = queryDesc->cplan; estate->es_part_prune_infos = plannedstmt->partPruneInfos; /* @@ -850,6 +904,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -917,6 +973,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) sp_eflags |= EXEC_FLAG_REWIND; subplanstate = ExecInitNode(subplan, estate, sp_eflags); + if (!ExecPlanStillValid(estate)) + goto failed; estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); @@ -930,6 +988,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -973,6 +1033,19 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such and release useless + * locks. + */ + queryDesc->plan_valid = false; + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, false); + /* Also ask ExecCloseRangeTableRelations() to release locks. */ + estate->es_top_eflags |= EXEC_FLAG_REL_LOCKS; } /* @@ -1389,7 +1462,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -1558,7 +1631,8 @@ ExecCloseResultRelations(EState *estate) /* * Close all relations opened by ExecGetRangeTableRelation(). * - * We do not release any locks we might hold on those rels. + * We do not release any locks we might hold on those rels, unless + * the caller asked otherwise. */ void ExecCloseRangeTableRelations(EState *estate) @@ -1567,8 +1641,12 @@ ExecCloseRangeTableRelations(EState *estate) for (i = 0; i < estate->es_range_table_size; i++) { + int lockmode = NoLock; + + if (estate->es_top_eflags & EXEC_FLAG_REL_LOCKS) + lockmode = exec_rt_fetch(i+1, estate)->rellockmode; if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + table_close(estate->es_relations[i], lockmode); } } @@ -2797,7 +2875,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2883,6 +2962,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..fe1d173501 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 651ad24fc1..a1bb1ac50f 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -1813,6 +1813,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1939,6 +1941,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..bfc4b6f81c 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return result; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -402,6 +405,8 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); + if (!ExecPlanStillValid(estate)) + return result; subps = lappend(subps, sstate); } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index c33a3c0bec..d5bd268514 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -800,7 +800,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -844,6 +845,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..949bdfc837 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 20d23696a5..94b7d08c93 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3295,6 +3295,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return aggstate; /* * initialize source tuple type. diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index cb25499b3f..2e0bfbe85a 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -148,6 +148,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return appendstate; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -218,6 +220,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return appendstate; } appendstate->as_first_partial_plan = firstvalid; diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..6b559bae2b 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return bitmapandstate; i++; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..a545018701 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -763,11 +763,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..87eb5dd5d3 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return bitmaporstate; i++; } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..efb94f9c59 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return css; css->ss.ss_currentRelation = scan_rel; } diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..c9a072e911 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * Tell the FDW to initialize the scan. diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..365d3af3e4 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gatherstate; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..8d2809f079 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gm_state; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..fa6dad3939 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return grpstate; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index eceee99374..6afc04edf1 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -379,6 +379,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return hashstate; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index b215e3f59a..0e2f931efa 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -659,8 +659,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..093c33d8ca 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return incrsortstate; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..a37a48c94a 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -512,6 +512,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..00dcb8424f 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -925,6 +925,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..2fcbde74ed 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return limitstate; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..3a8aa2b5a4 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return lrstate; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..09982fd38c 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return matstate; /* * Initialize result type and slot. No need to initialize projection info diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 74f7d21bc8..ad7a1f6fe0 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -931,6 +931,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mstate; /* * Initialize return slot and type. No need to initialize projection info diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..c3fdddecc5 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -96,6 +96,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return mergestate; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -152,6 +154,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; } mergestate->ps.ps_ProjInfo = NULL; diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..489c651a25 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return mergestate; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 1ac65172e4..27dda57c3d 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -4010,6 +4010,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + return mtstate; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -4036,6 +4039,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + return mtstate; /* * For child result relations, store the root result relation @@ -4063,6 +4068,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mtstate; /* * Do additional per-result-relation initialization. diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..299f6b3a57 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; /* * Initialize result slot, type and projection. diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..b85ba2cf23 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return state; /* * we don't use inner plan diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..967fe4f287 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..a79d407fa8 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return resstate; /* * we don't use inner plan diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..31a6148977 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..88fe4d40d5 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..697dc699a5 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return setopstate; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..c8ed534f29 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return sortstate; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..3bb8bbbb84 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return subquerystate; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..c528a63c38 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidrangestate; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index fe6a964ee1..a8e449e70a 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidstate; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..6b183d7324 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return uniquestate; /* * Initialize result slot and type. Unique nodes do no projections, so diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index d61d57e9a8..239ad14dfc 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -2450,6 +2450,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return winstate; /* * initialize source tuple type (which is also the tuple type that we'll diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index 61f03e3999..38d76c6719 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,8 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if *replan is set. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1777,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2548,6 +2556,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2657,6 +2666,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2664,14 +2674,29 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2846,10 +2871,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2893,14 +2917,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile index af12c64878..7fb0d2d202 100644 --- a/src/backend/nodes/Makefile +++ b/src/backend/nodes/Makefile @@ -52,6 +52,7 @@ node_headers = \ access/tsmapi.h \ commands/event_trigger.h \ commands/trigger.h \ + executor/execdesc.h \ executor/tuptable.h \ foreign/fdwapi.h \ nodes/bitmapset.h \ diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl index 19ed29657c..69e60206ba 100644 --- a/src/backend/nodes/gen_node_support.pl +++ b/src/backend/nodes/gen_node_support.pl @@ -63,6 +63,7 @@ my @all_input_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/bitmapset.h @@ -87,6 +88,7 @@ my @nodetag_only_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/lockoptions.h diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index db5ff6fdca..670eba3a3a 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 186fc8014b..454e30e0ca 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -599,6 +600,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index c74bac20b1..29d13e95db 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1834,11 +1834,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..34d3f4ff8d 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1196,6 +1196,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1700,6 +1701,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1995,6 +1997,12 @@ exec_bind_message(StringInfo input_message) */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..cf3a9790d6 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -75,8 +71,10 @@ CreateQueryDesc(PlannedStmt *plannedstmt, { QueryDesc *qd = (QueryDesc *) palloc(sizeof(QueryDesc)); + qd->type = T_QueryDesc; qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +114,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +345,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +354,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +366,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +388,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +411,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +420,50 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +471,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +495,69 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool pushed_active_snapshot = false; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* Must set snapshot before starting executor. */ + if (!pushed_active_snapshot && !is_utility) + { + PushActiveSnapshot(GetTransactionSnapshot()); + pushed_active_snapshot = true; + } + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + pushed_active_snapshot ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalMultiRun() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + if (!is_utility) + { + ExecutorStart(queryDesc, 0); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + if (pushed_active_snapshot) + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } + } + } + + if (pushed_active_snapshot) + PopActiveSnapshot(); + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +569,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1167,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1188,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = lfirst_node(QueryDesc, qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1241,7 +1216,7 @@ PortalRunMulti(Portal portal, */ if (!active_snapshot_set) { - Snapshot snapshot = GetTransactionSnapshot(); + Snapshot snapshot = qdesc->snapshot; /* If told to, register the snapshot and save in portal */ if (setHoldSnapshot) @@ -1271,23 +1246,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1336,19 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + /* portal->queryDesc is free'd by PortalCleanup(). */ + if (qdesc != portal->queryDesc) + { + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); + } } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c07382051d..38ae43e24b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..da39b2e4ff 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +107,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index e7e25c057e..15a1abaacf 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -59,6 +60,10 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should ExecGetRangeTableRelation + * lock relations? */ +#define EXEC_FLAG_REL_LOCKS 0x8000 /* should ExecCloseRangeTableRelations + * release locks? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -245,6 +250,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan ? + CachedPlanStillValid(estate->es_cachedplan) : true; +} /* ---------------------------------------------------------------- * ExecProcNode diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 20f4c8b35f..89f5a627c8 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build index efe0834afb..a8fdd9e176 100644 --- a/src/include/nodes/meson.build +++ b/src/include/nodes/meson.build @@ -13,6 +13,7 @@ node_support_input_i = [ 'access/tsmapi.h', 'commands/event_trigger.h', 'commands/trigger.h', + 'executor/execdesc.h', 'executor/tuptable.h', 'foreign/fdwapi.h', 'nodes/bitmapset.h', diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index 0d4b1ec4e4..71004fee75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 4781a9c632..da9e73fb16 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..c2e485ac2c 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,21 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor after it has finished taking locks on a plan tree + * in a CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + +extern bool CachedPlanStillValid(CachedPlan *cplan); extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-02-03 13:01 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-02-03 13:01 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers On Thu, Feb 2, 2023 at 11:49 PM Amit Langote <[email protected]> wrote: > On Fri, Jan 27, 2023 at 4:01 PM Amit Langote <[email protected]> wrote: > > I didn't actually go with calling the plancache on every lock taken on > > a relation, that is, in ExecGetRangeTableRelation(). One thing about > > doing it that way that I didn't quite like (or didn't see a clean > > enough way to code) is the need to complicate the ExecInitNode() > > traversal for handling the abrupt suspension of the ongoing setup of > > the PlanState tree. > > OK, I gave this one more try and attached is what I came up with. > > This adds a ExecPlanStillValid(), which is called right after anything > that may in turn call ExecGetRangeTableRelation() which has been > taught to lock a relation if EXEC_FLAG_GET_LOCKS has been passed in > EState.es_top_eflags. That includes all ExecInitNode() calls, and a > few other functions that call ExecGetRangeTableRelation() directly, > such as ExecOpenScanRelation(). If ExecPlanStillValid() returns > false, that is, if EState.es_cachedplan is found to have been > invalidated after a lock being taken by ExecGetRangeTableRelation(), > whatever funcion called it must return immediately and so must its > caller and so on. ExecEndPlan() seems to be able to clean up after a > partially finished attempt of initializing a PlanState tree in this > way. Maybe my preliminary testing didn't catch cases where pointers > to resources that are normally put into the nodes of a PlanState tree > are now left dangling, because a partially built PlanState tree is not > accessible to ExecEndPlan; QueryDesc.planstate would remain NULL in > such cases. Maybe there's only es_tupleTable and es_relations that > needs to be explicitly released and the rest is taken care of by > resetting the ExecutorState context. In the attached updated patch, I've made the functions that check ExecPlanStillValid() to return NULL (if returning something) instead of returning partially initialized structs. Those partially initialized structs were not being subsequently looked at anyway. > On testing, I'm afraid we're going to need something like > src/test/modules/delay_execution to test that concurrent changes to > relation(s) in PlannedStmt.relationOids that occur somewhere between > RevalidateCachedQuery() and InitPlan() result in the latter to be > aborted and that it is handled correctly. It seems like it is only > the locking of partitions (that are not present in an unplanned Query > and thus not protected by AcquirePlannerLocks()) that can trigger > replanning of a CachedPlan, so any tests we write should involve > partitions. Should this try to test as many plan shapes as possible > though given the uncertainty around ExecEndPlan() robustness or should > manual auditing suffice to be sure that nothing's broken? I've added a test case under src/modules/delay_execution by adding a new ExecutorStart_hook that works similarly as delay_execution_planner(). The test works by allowing a concurrent session to drop an object being referenced in a cached plan being initialized while the ExecutorStart_hook waits to get an advisory lock. The concurrent drop of the referenced object is detected during ExecInitNode() and thus triggers replanning of the cached plan. I also fixed a bug in the ExplainExecuteQuery() while testing and some comments. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v33-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch (96.5K, 2-v33-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 4a8e2b6c4d87e0ae81becd74b4a1e4d5217eb05a Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v33] Move AcquireExecutorLocks()'s responsibility into the executor --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 142 ++++++--- src/backend/commands/extension.c | 1 + src/backend/commands/matview.c | 2 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 30 +- src/backend/executor/execMain.c | 105 ++++++- src/backend/executor/execParallel.c | 7 +- src/backend/executor/execPartition.c | 4 + src/backend/executor/execProcnode.c | 5 + src/backend/executor/execUtils.c | 5 +- src/backend/executor/functions.c | 1 + src/backend/executor/nodeAgg.c | 2 + src/backend/executor/nodeAppend.c | 4 + src/backend/executor/nodeBitmapAnd.c | 2 + src/backend/executor/nodeBitmapHeapscan.c | 4 + src/backend/executor/nodeBitmapOr.c | 2 + src/backend/executor/nodeCustom.c | 2 + src/backend/executor/nodeForeignscan.c | 4 + src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 2 + src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 4 + src/backend/executor/nodeIncrementalSort.c | 2 + src/backend/executor/nodeIndexonlyscan.c | 2 + src/backend/executor/nodeIndexscan.c | 2 + src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 2 + src/backend/executor/nodeMemoize.c | 2 + src/backend/executor/nodeMergeAppend.c | 4 + src/backend/executor/nodeMergejoin.c | 4 + src/backend/executor/nodeModifyTable.c | 7 + src/backend/executor/nodeNestloop.c | 4 + src/backend/executor/nodeProjectSet.c | 2 + src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 2 + src/backend/executor/nodeSamplescan.c | 2 + src/backend/executor/nodeSeqscan.c | 2 + src/backend/executor/nodeSetOp.c | 2 + src/backend/executor/nodeSort.c | 2 + src/backend/executor/nodeSubqueryscan.c | 2 + src/backend/executor/nodeTidrangescan.c | 2 + src/backend/executor/nodeTidscan.c | 2 + src/backend/executor/nodeUnique.c | 2 + src/backend/executor/nodeWindowAgg.c | 2 + src/backend/executor/spi.c | 44 ++- src/backend/nodes/Makefile | 1 + src/backend/nodes/gen_node_support.pl | 2 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 8 + src/backend/tcop/pquery.c | 291 +++++++++--------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 +++----- src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 12 + src/include/nodes/execnodes.h | 2 + src/include/nodes/meson.build | 1 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 15 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 43 +++ .../specs/cached-plan-replan.spec | 39 +++ 76 files changed, 846 insertions(+), 340 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index 8043b4e9b1..a438c547e8 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index fbbf28cf06..8fdc966a73 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,89 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,29 +593,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - - Assert(plannedstmt->commandType != CMD_UTILITY); - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -546,38 +611,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4851,6 +4884,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index b1509cc505..e2f79cc7a7 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -780,6 +780,7 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..17d457ccfb 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,7 +409,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..3099536a54 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,17 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, + * it must be recreated if *replan is set. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +582,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +626,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +648,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index a5115b9c1f..d97c9de409 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,27 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan. Locks necessary + * to validate such a plan tree must be taken when initializing the plan tree + * in InitPlan(), so this sets the eflag EXEC_FLAG_GET_LOCKS. If the + * CachedPlan gets invalidated as these locks are taken, InitPlan returns + * without setting queryDesc->planstate and sets queryDesc->plan_valid to + * false. Caller must retry the execution with a freshly created CachedPlan + * in that case. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + CachedPlanStillValid(queryDesc->cplan)); + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +598,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +811,43 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } +/* + * Lock view relations in a given query's range table. + */ +static void +ExecLockViewRelations(List *viewRelations, EState *estate, bool acquire) +{ + ListCell *lc; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + if (acquire) + LockRelationOid(rte->relid, rte->rellockmode); + else + UnlockRelationOid(rte->relid, rte->rellockmode); + } +} /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -807,17 +864,21 @@ InitPlan(QueryDesc *queryDesc, int eflags) int i; /* - * Do permissions checks and save the list for later use. + * initialize the node's execution state */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); - estate->es_rteperminfos = plannedstmt->permInfos; + ExecInitRangeTable(estate, rangeTable); + + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, true); /* - * initialize the node's execution state + * Do permissions checks and save the list for later use. */ - ExecInitRangeTable(estate, rangeTable); + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + estate->es_rteperminfos = plannedstmt->permInfos; estate->es_plannedstmt = plannedstmt; + estate->es_cachedplan = queryDesc->cplan; estate->es_part_prune_infos = plannedstmt->partPruneInfos; /* @@ -850,6 +911,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -917,6 +980,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) sp_eflags |= EXEC_FLAG_REWIND; subplanstate = ExecInitNode(subplan, estate, sp_eflags); + if (!ExecPlanStillValid(estate)) + goto failed; estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); @@ -930,6 +995,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -973,6 +1040,19 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such and release useless + * locks. + */ + queryDesc->plan_valid = false; + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, false); + /* Also ask ExecCloseRangeTableRelations() to release locks. */ + estate->es_top_eflags |= EXEC_FLAG_REL_LOCKS; } /* @@ -1389,7 +1469,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -1558,7 +1638,8 @@ ExecCloseResultRelations(EState *estate) /* * Close all relations opened by ExecGetRangeTableRelation(). * - * We do not release any locks we might hold on those rels. + * We do not release any locks we might hold on those rels, unless + * the caller asked otherwise. */ void ExecCloseRangeTableRelations(EState *estate) @@ -1567,8 +1648,12 @@ ExecCloseRangeTableRelations(EState *estate) for (i = 0; i < estate->es_range_table_size; i++) { + int lockmode = NoLock; + + if (estate->es_top_eflags & EXEC_FLAG_REL_LOCKS) + lockmode = exec_rt_fetch(i+1, estate)->rellockmode; if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + table_close(estate->es_relations[i], lockmode); } } @@ -2797,7 +2882,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2883,6 +2969,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..fe1d173501 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 651ad24fc1..a1bb1ac50f 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -1813,6 +1813,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1939,6 +1941,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..bd0c2cba92 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return NULL; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -402,6 +405,8 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); + if (!ExecPlanStillValid(estate)) + return NULL; subps = lappend(subps, sstate); } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index c33a3c0bec..d5bd268514 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -800,7 +800,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -844,6 +845,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..949bdfc837 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 20d23696a5..f9b668dc01 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3295,6 +3295,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize source tuple type. diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index cb25499b3f..fd0ad98621 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -148,6 +148,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return NULL; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -218,6 +220,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; } appendstate->as_first_partial_plan = firstvalid; diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..98cbeb2502 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; i++; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..121b1afa5d 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -763,11 +763,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..be736946f1 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; i++; } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..91239cc500 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; css->ss.ss_currentRelation = scan_rel; } diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..f130d5863d 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Tell the FDW to initialize the scan. diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..4a7715b8cc 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..9e383c96ff 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..87af2a92f9 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index eceee99374..c8fedee777 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -379,6 +379,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index b215e3f59a..86420e8f17 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -659,8 +659,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..0456ad779f 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..e0aaeb5ebd 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -512,6 +512,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..5090ee39e0 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -925,6 +925,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..d8789553e1 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..9104954bb1 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..6ef50d3960 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result type and slot. No need to initialize projection info diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 74f7d21bc8..4ecc60a238 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -931,6 +931,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize return slot and type. No need to initialize projection info diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..b12a02c028 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -96,6 +96,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return NULL; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -152,6 +154,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; } mergestate->ps.ps_ProjInfo = NULL; diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..0157a7ff3c 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return NULL; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 1ac65172e4..355964d103 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -4010,6 +4010,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + return NULL; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -4036,6 +4039,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + return NULL; /* * For child result relations, store the root result relation @@ -4063,6 +4068,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Do additional per-result-relation initialization. diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..e4319f5c90 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result slot, type and projection. diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..a168cd68f6 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * we don't use inner plan diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..3dae9b1497 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..9da456be4a 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * we don't use inner plan diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..22357e7a0e 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..b0b34cd14e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..2c350e6c24 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..216a5afb40 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..34afe14bea 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..613b377c7c 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index fe6a964ee1..23782aad89 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..06257e9e51 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result slot and type. Unique nodes do no projections, so diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index d61d57e9a8..d8a9f1e94e 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -2450,6 +2450,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize source tuple type (which is also the tuple type that we'll diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index 61f03e3999..38d76c6719 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,8 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if *replan is set. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1777,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2548,6 +2556,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2657,6 +2666,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2664,14 +2674,29 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2846,10 +2871,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2893,14 +2917,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile index af12c64878..7fb0d2d202 100644 --- a/src/backend/nodes/Makefile +++ b/src/backend/nodes/Makefile @@ -52,6 +52,7 @@ node_headers = \ access/tsmapi.h \ commands/event_trigger.h \ commands/trigger.h \ + executor/execdesc.h \ executor/tuptable.h \ foreign/fdwapi.h \ nodes/bitmapset.h \ diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl index 19ed29657c..69e60206ba 100644 --- a/src/backend/nodes/gen_node_support.pl +++ b/src/backend/nodes/gen_node_support.pl @@ -63,6 +63,7 @@ my @all_input_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/bitmapset.h @@ -87,6 +88,7 @@ my @nodetag_only_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/lockoptions.h diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index db5ff6fdca..670eba3a3a 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 186fc8014b..454e30e0ca 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -599,6 +600,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index c74bac20b1..29d13e95db 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1834,11 +1834,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 470b734e9e..34d3f4ff8d 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1196,6 +1196,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1700,6 +1701,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1995,6 +1997,12 @@ exec_bind_message(StringInfo input_message) */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..cf3a9790d6 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -75,8 +71,10 @@ CreateQueryDesc(PlannedStmt *plannedstmt, { QueryDesc *qd = (QueryDesc *) palloc(sizeof(QueryDesc)); + qd->type = T_QueryDesc; qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +114,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +345,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +354,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +366,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +388,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +411,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +420,50 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +471,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +495,69 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool pushed_active_snapshot = false; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* Must set snapshot before starting executor. */ + if (!pushed_active_snapshot && !is_utility) + { + PushActiveSnapshot(GetTransactionSnapshot()); + pushed_active_snapshot = true; + } + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + pushed_active_snapshot ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalMultiRun() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + if (!is_utility) + { + ExecutorStart(queryDesc, 0); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + if (pushed_active_snapshot) + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } + } + } + + if (pushed_active_snapshot) + PopActiveSnapshot(); + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +569,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1167,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1188,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = lfirst_node(QueryDesc, qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1241,7 +1216,7 @@ PortalRunMulti(Portal portal, */ if (!active_snapshot_set) { - Snapshot snapshot = GetTransactionSnapshot(); + Snapshot snapshot = qdesc->snapshot; /* If told to, register the snapshot and save in portal */ if (setHoldSnapshot) @@ -1271,23 +1246,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1336,19 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + /* portal->queryDesc is free'd by PortalCleanup(). */ + if (qdesc != portal->queryDesc) + { + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); + } } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c07382051d..38ae43e24b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..da39b2e4ff 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +107,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index e7e25c057e..8c680358e8 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -59,6 +60,10 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should ExecGetRangeTableRelation + * lock relations? */ +#define EXEC_FLAG_REL_LOCKS 0x8000 /* should ExecCloseRangeTableRelations + * release locks? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -245,6 +250,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 20f4c8b35f..89f5a627c8 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build index efe0834afb..a8fdd9e176 100644 --- a/src/include/nodes/meson.build +++ b/src/include/nodes/meson.build @@ -13,6 +13,7 @@ node_support_input_i = [ 'access/tsmapi.h', 'commands/event_trigger.h', 'commands/trigger.h', + 'executor/execdesc.h', 'executor/tuptable.h', 'foreign/fdwapi.h', 'nodes/bitmapset.h', diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index 0d4b1ec4e4..71004fee75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 4781a9c632..da9e73fb16 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..c2e485ac2c 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,21 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor after it has finished taking locks on a plan tree + * in a CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + +extern bool CachedPlanStillValid(CachedPlan *cplan); extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..eaac55122b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,43 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +--------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo1 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo1_a + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo1_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo1 foo_1 + Filter: (a = $1) +(4 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..5bd5fdbf1c --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,39 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1); + CREATE INDEX foo1_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo1_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-02-07 18:08 Andres Freund <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Andres Freund @ 2023-02-07 18:08 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; David Rowley <[email protected]>; pgsql-hackers Hi, On 2023-02-03 22:01:09 +0900, Amit Langote wrote: > I've added a test case under src/modules/delay_execution by adding a > new ExecutorStart_hook that works similarly as > delay_execution_planner(). The test works by allowing a concurrent > session to drop an object being referenced in a cached plan being > initialized while the ExecutorStart_hook waits to get an advisory > lock. The concurrent drop of the referenced object is detected during > ExecInitNode() and thus triggers replanning of the cached plan. > > I also fixed a bug in the ExplainExecuteQuery() while testing and some comments. The tests seem to frequently hang on freebsd: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3478 Greetings, Andres Freund ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-02-08 10:31 Amit Langote <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-02-08 10:31 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> On Tue, Feb 7, 2023 at 23:38 Andres Freund <[email protected]> wrote: > Hi, > > On 2023-02-03 22:01:09 +0900, Amit Langote wrote: > > I've added a test case under src/modules/delay_execution by adding a > > new ExecutorStart_hook that works similarly as > > delay_execution_planner(). The test works by allowing a concurrent > > session to drop an object being referenced in a cached plan being > > initialized while the ExecutorStart_hook waits to get an advisory > > lock. The concurrent drop of the referenced object is detected during > > ExecInitNode() and thus triggers replanning of the cached plan. > > > > I also fixed a bug in the ExplainExecuteQuery() while testing and some > comments. > > The tests seem to frequently hang on freebsd: > > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3478 Thanks for the heads up. I’ve noticed this one too, though couldn’t find the testrun artifacts like I could get for some other failures (on other cirrus machines). Has anyone else been a similar situation? > > <https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3478; -- Thanks, Amit Langote EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-03-02 13:52 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-03-02 13:52 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> On Wed, Feb 8, 2023 at 7:31 PM Amit Langote <[email protected]> wrote: > On Tue, Feb 7, 2023 at 23:38 Andres Freund <[email protected]> wrote: >> The tests seem to frequently hang on freebsd: >> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3478 > > Thanks for the heads up. I’ve noticed this one too, though couldn’t find the testrun artifacts like I could get for some other failures (on other cirrus machines). Has anyone else been a similar situation? I think I have figured out what might be going wrong on that cfbot animal after building with the same CPPFLAGS as that animal locally. I had forgotten to update _out/_readRangeTblEntry() to account for the patch's change that a view's RTE_SUBQUERY now also preserves relkind in addition to relid and rellockmode for the locking consideration. Also, I noticed that a multi-query Portal execution with rules was failing (thanks to a regression test added in a7d71c41db) because of the snapshot used for the 2nd query onward not being updated for command ID change under patched model of multi-query Portal execution. To wit, under the patched model, all queries in the multi-query Portal case undergo ExecutorStart() before any of it is run with ExecutorRun(). The patch hadn't changed things however to update the snapshot's command ID for the 2nd query onwards, which caused the aforementioned test case to fail. This new model does however mean that the 2nd query onwards must use PushCopiedSnapshot() given the current requirement of UpdateActiveSnapshotCommandId() that the snapshot passed to it must not be referenced anywhere else. The new model basically requires that each query's QueryDesc points to its own copy of the ActiveSnapshot. That may not be a thing in favor of the patched model though. For now, I haven't been able to come up with a better alternative. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v34-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch (97.9K, 2-v34-0001-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 1e8b5333a9619ec87c3e31c2034422309a4719ad Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v34] Move AcquireExecutorLocks()'s responsibility into the executor --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 142 +++++--- src/backend/commands/extension.c | 1 + src/backend/commands/matview.c | 2 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 30 +- src/backend/executor/execMain.c | 105 +++++- src/backend/executor/execParallel.c | 7 +- src/backend/executor/execPartition.c | 4 + src/backend/executor/execProcnode.c | 5 + src/backend/executor/execUtils.c | 5 +- src/backend/executor/functions.c | 1 + src/backend/executor/nodeAgg.c | 2 + src/backend/executor/nodeAppend.c | 4 + src/backend/executor/nodeBitmapAnd.c | 2 + src/backend/executor/nodeBitmapHeapscan.c | 4 + src/backend/executor/nodeBitmapOr.c | 2 + src/backend/executor/nodeCustom.c | 2 + src/backend/executor/nodeForeignscan.c | 4 + src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 2 + src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 4 + src/backend/executor/nodeIncrementalSort.c | 2 + src/backend/executor/nodeIndexonlyscan.c | 2 + src/backend/executor/nodeIndexscan.c | 2 + src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 2 + src/backend/executor/nodeMemoize.c | 2 + src/backend/executor/nodeMergeAppend.c | 4 + src/backend/executor/nodeMergejoin.c | 4 + src/backend/executor/nodeModifyTable.c | 7 + src/backend/executor/nodeNestloop.c | 4 + src/backend/executor/nodeProjectSet.c | 2 + src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 2 + src/backend/executor/nodeSamplescan.c | 2 + src/backend/executor/nodeSeqscan.c | 2 + src/backend/executor/nodeSetOp.c | 2 + src/backend/executor/nodeSort.c | 2 + src/backend/executor/nodeSubqueryscan.c | 2 + src/backend/executor/nodeTidrangescan.c | 2 + src/backend/executor/nodeTidscan.c | 2 + src/backend/executor/nodeUnique.c | 2 + src/backend/executor/nodeWindowAgg.c | 2 + src/backend/executor/spi.c | 46 ++- src/backend/nodes/Makefile | 1 + src/backend/nodes/gen_node_support.pl | 2 + src/backend/nodes/outfuncs.c | 1 + src/backend/nodes/readfuncs.c | 1 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 8 + src/backend/tcop/pquery.c | 307 +++++++++--------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 ++------ src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 12 + src/include/nodes/execnodes.h | 2 + src/include/nodes/meson.build | 1 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 15 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 43 +++ .../specs/cached-plan-replan.spec | 39 +++ 78 files changed, 866 insertions(+), 340 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index 8043b4e9b1..a438c547e8 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index e57bda7b62..e56ccdca66 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,89 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,29 +593,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - - Assert(plannedstmt->commandType != CMD_UTILITY); - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -546,38 +611,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4851,6 +4884,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index b1509cc505..e2f79cc7a7 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -780,6 +780,7 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..17d457ccfb 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,7 +409,7 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..3099536a54 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,17 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, + * it must be recreated if *replan is set. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +582,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +626,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +648,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 39bfb48dc2..dafdd8a783 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,27 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan. Locks necessary + * to validate such a plan tree must be taken when initializing the plan tree + * in InitPlan(), so this sets the eflag EXEC_FLAG_GET_LOCKS. If the + * CachedPlan gets invalidated as these locks are taken, InitPlan returns + * without setting queryDesc->planstate and sets queryDesc->plan_valid to + * false. Caller must retry the execution with a freshly created CachedPlan + * in that case. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + CachedPlanStillValid(queryDesc->cplan)); + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +598,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +811,43 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } +/* + * Lock view relations in a given query's range table. + */ +static void +ExecLockViewRelations(List *viewRelations, EState *estate, bool acquire) +{ + ListCell *lc; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + if (acquire) + LockRelationOid(rte->relid, rte->rellockmode); + else + UnlockRelationOid(rte->relid, rte->rellockmode); + } +} /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -807,17 +864,21 @@ InitPlan(QueryDesc *queryDesc, int eflags) int i; /* - * Do permissions checks and save the list for later use. + * initialize the node's execution state */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); - estate->es_rteperminfos = plannedstmt->permInfos; + ExecInitRangeTable(estate, rangeTable); + + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, true); /* - * initialize the node's execution state + * Do permissions checks and save the list for later use. */ - ExecInitRangeTable(estate, rangeTable); + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + estate->es_rteperminfos = plannedstmt->permInfos; estate->es_plannedstmt = plannedstmt; + estate->es_cachedplan = queryDesc->cplan; estate->es_part_prune_infos = plannedstmt->partPruneInfos; /* @@ -850,6 +911,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -917,6 +980,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) sp_eflags |= EXEC_FLAG_REWIND; subplanstate = ExecInitNode(subplan, estate, sp_eflags); + if (!ExecPlanStillValid(estate)) + goto failed; estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); @@ -930,6 +995,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -973,6 +1040,19 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such and release useless + * locks. + */ + queryDesc->plan_valid = false; + if (eflags & EXEC_FLAG_GET_LOCKS) + ExecLockViewRelations(plannedstmt->viewRelations, estate, false); + /* Also ask ExecCloseRangeTableRelations() to release locks. */ + estate->es_top_eflags |= EXEC_FLAG_REL_LOCKS; } /* @@ -1389,7 +1469,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -1558,7 +1638,8 @@ ExecCloseResultRelations(EState *estate) /* * Close all relations opened by ExecGetRangeTableRelation(). * - * We do not release any locks we might hold on those rels. + * We do not release any locks we might hold on those rels, unless + * the caller asked otherwise. */ void ExecCloseRangeTableRelations(EState *estate) @@ -1567,8 +1648,12 @@ ExecCloseRangeTableRelations(EState *estate) for (i = 0; i < estate->es_range_table_size; i++) { + int lockmode = NoLock; + + if (estate->es_top_eflags & EXEC_FLAG_REL_LOCKS) + lockmode = exec_rt_fetch(i+1, estate)->rellockmode; if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + table_close(estate->es_relations[i], lockmode); } } @@ -2797,7 +2882,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2883,6 +2969,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..fe1d173501 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index fd6ca8a5d9..ae6a974e7a 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -1817,6 +1817,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1943,6 +1945,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..bd0c2cba92 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return NULL; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -402,6 +405,8 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); + if (!ExecPlanStillValid(estate)) + return NULL; subps = lappend(subps, sstate); } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index c33a3c0bec..d5bd268514 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -800,7 +800,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -844,6 +845,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..949bdfc837 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 20d23696a5..f9b668dc01 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3295,6 +3295,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize source tuple type. diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index cb25499b3f..fd0ad98621 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -148,6 +148,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return NULL; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -218,6 +220,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; } appendstate->as_first_partial_plan = firstvalid; diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..98cbeb2502 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -89,6 +89,8 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; i++; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..121b1afa5d 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -763,11 +763,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..be736946f1 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -90,6 +90,8 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; i++; } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..91239cc500 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; css->ss.ss_currentRelation = scan_rel; } diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..f130d5863d 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Tell the FDW to initialize the scan. diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..4a7715b8cc 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..9e383c96ff 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..87af2a92f9 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index eceee99374..c8fedee777 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -379,6 +379,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index b215e3f59a..86420e8f17 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -659,8 +659,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..0456ad779f 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..e0aaeb5ebd 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -512,6 +512,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..5090ee39e0 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -925,6 +925,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..d8789553e1 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..9104954bb1 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..6ef50d3960 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result type and slot. No need to initialize projection info diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 74f7d21bc8..4ecc60a238 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -931,6 +931,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize return slot and type. No need to initialize projection info diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..b12a02c028 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -96,6 +96,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return NULL; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -152,6 +154,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; } mergestate->ps.ps_ProjInfo = NULL; diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..0157a7ff3c 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return NULL; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 6f0543af83..66c0ebe16d 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -4006,6 +4006,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + return NULL; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -4032,6 +4035,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + return NULL; /* * For child result relations, store the root result relation @@ -4059,6 +4064,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Do additional per-result-relation initialization. diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..e4319f5c90 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result slot, type and projection. diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..a168cd68f6 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * we don't use inner plan diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..3dae9b1497 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..9da456be4a 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * we don't use inner plan diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..22357e7a0e 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..b0b34cd14e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..2c350e6c24 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..216a5afb40 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type. diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..34afe14bea 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..613b377c7c 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -386,6 +386,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 862bd0330b..1b0a2d8083 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -529,6 +529,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..06257e9e51 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Initialize result slot and type. Unique nodes do no projections, so diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 7c07fb0684..7363291023 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -2451,6 +2451,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return NULL; /* * initialize source tuple type (which is also the tuple type that we'll diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index e3a170c38b..edea7675d4 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,8 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if *replan is set. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1777,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2552,6 +2560,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2661,6 +2670,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2668,14 +2678,31 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + Assert(cplan); + ReleaseCachedPlan(cplan, plan_owner); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2850,10 +2877,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2897,14 +2923,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile index af12c64878..7fb0d2d202 100644 --- a/src/backend/nodes/Makefile +++ b/src/backend/nodes/Makefile @@ -52,6 +52,7 @@ node_headers = \ access/tsmapi.h \ commands/event_trigger.h \ commands/trigger.h \ + executor/execdesc.h \ executor/tuptable.h \ foreign/fdwapi.h \ nodes/bitmapset.h \ diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl index ecbcadb8bf..b1cf61c0a2 100644 --- a/src/backend/nodes/gen_node_support.pl +++ b/src/backend/nodes/gen_node_support.pl @@ -63,6 +63,7 @@ my @all_input_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/bitmapset.h @@ -87,6 +88,7 @@ my @nodetag_only_files = qw( access/tsmapi.h commands/event_trigger.h commands/trigger.h + executor/execdesc.h executor/tuptable.h foreign/fdwapi.h nodes/lockoptions.h diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index ba00b99249..955286513d 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -513,6 +513,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ WRITE_OID_FIELD(relid); + WRITE_CHAR_FIELD(relkind); WRITE_INT_FIELD(rellockmode); WRITE_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index f3629cdfd1..3bc5a6dca0 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -480,6 +480,7 @@ _readRangeTblEntry(void) READ_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ READ_OID_FIELD(relid); + READ_CHAR_FIELD(relkind); READ_INT_FIELD(rellockmode); READ_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index a1873ce26d..271d2539e8 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 5cc8366af6..f13240bf33 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -604,6 +605,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index a614e3f5bd..de07e53178 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1847,11 +1847,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..a291bcfcfc 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1199,6 +1199,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1703,6 +1704,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1998,6 +2000,12 @@ exec_bind_message(StringInfo input_message) */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..b9df5d4a04 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -75,8 +71,10 @@ CreateQueryDesc(PlannedStmt *plannedstmt, { QueryDesc *qd = (QueryDesc *) palloc(sizeof(QueryDesc)); + qd->type = T_QueryDesc; qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +114,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +345,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +354,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +366,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +388,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +411,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +420,50 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +471,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +495,85 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool first = true; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* + * Push the snapshot to be used by the executor. + */ + if (!is_utility) + { + /* + * Must copy the snapshot if we'll need to update + * its command ID. + */ + if (!first) + PushCopiedSnapshot(GetTransactionSnapshot()); + else + PushActiveSnapshot(GetTransactionSnapshot()); + } + + /* + * From the 2nd statement onwards, update the command + * ID and the snapshot to match. + */ + if (!first) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + first = false; + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + !is_utility ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalMultiRun() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + if (is_utility) + continue; + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + ExecutorStart(queryDesc, 0); + PopActiveSnapshot(); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + portal->plan_valid = false; + goto early_exit; + } + } + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +585,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1183,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1204,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = lfirst_node(QueryDesc, qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1241,7 +1232,7 @@ PortalRunMulti(Portal portal, */ if (!active_snapshot_set) { - Snapshot snapshot = GetTransactionSnapshot(); + Snapshot snapshot = qdesc->snapshot; /* If told to, register the snapshot and save in portal */ if (setHoldSnapshot) @@ -1271,23 +1262,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1352,19 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + /* portal->queryDesc is free'd by PortalCleanup(). */ + if (qdesc != portal->queryDesc) + { + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); + } } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c07382051d..38ae43e24b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..da39b2e4ff 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +107,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index e7e25c057e..8c680358e8 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -59,6 +60,10 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should ExecGetRangeTableRelation + * lock relations? */ +#define EXEC_FLAG_REL_LOCKS 0x8000 /* should ExecCloseRangeTableRelations + * release locks? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -245,6 +250,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 20f4c8b35f..89f5a627c8 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build index efe0834afb..a8fdd9e176 100644 --- a/src/include/nodes/meson.build +++ b/src/include/nodes/meson.build @@ -13,6 +13,7 @@ node_support_input_i = [ 'access/tsmapi.h', 'commands/event_trigger.h', 'commands/trigger.h', + 'executor/execdesc.h', 'executor/tuptable.h', 'foreign/fdwapi.h', 'nodes/bitmapset.h', diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d61a62da19..9b888b0d75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 659bd05c0c..496410198f 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..c2e485ac2c 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,21 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor after it has finished taking locks on a plan tree + * in a CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + +extern bool CachedPlanStillValid(CachedPlan *cplan); extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..eaac55122b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,43 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +--------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo1 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo1_a + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo1_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo1 foo_1 + Filter: (a = $1) +(4 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..5bd5fdbf1c --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,39 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1); + CREATE INDEX foo1_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo1_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-03-14 10:07 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-03-14 10:07 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> On Thu, Mar 2, 2023 at 10:52 PM Amit Langote <[email protected]> wrote: > I think I have figured out what might be going wrong on that cfbot > animal after building with the same CPPFLAGS as that animal locally. > I had forgotten to update _out/_readRangeTblEntry() to account for the > patch's change that a view's RTE_SUBQUERY now also preserves relkind > in addition to relid and rellockmode for the locking consideration. > > Also, I noticed that a multi-query Portal execution with rules was > failing (thanks to a regression test added in a7d71c41db) because of > the snapshot used for the 2nd query onward not being updated for > command ID change under patched model of multi-query Portal execution. > To wit, under the patched model, all queries in the multi-query Portal > case undergo ExecutorStart() before any of it is run with > ExecutorRun(). The patch hadn't changed things however to update the > snapshot's command ID for the 2nd query onwards, which caused the > aforementioned test case to fail. > > This new model does however mean that the 2nd query onwards must use > PushCopiedSnapshot() given the current requirement of > UpdateActiveSnapshotCommandId() that the snapshot passed to it must > not be referenced anywhere else. The new model basically requires > that each query's QueryDesc points to its own copy of the > ActiveSnapshot. That may not be a thing in favor of the patched model > though. For now, I haven't been able to come up with a better > alternative. Here's a new version addressing the following 2 points. * Like views, I realized that non-leaf relations of partition trees scanned by an Append/MergeAppend would need to be locked separately, because ExecInitNode() traversal of the plan tree would not account for them. That is, they are not opened using ExecGetRangeTableRelation() or ExecOpenScanRelation(). One exception is that some (if not all) of those non-leaf relations may be referenced in PartitionPruneInfo and so locked as part of initializing the corresponding PartitionPruneState, but I decided not to complicate the code to filter out such relations from the set locked separately. To carry the set of relations to lock, the refactoring patch 0001 re-introduces the List of Bitmapset field named allpartrelids into Append/MergeAppend nodes, which we had previously removed on the grounds that those relations need not be locked separately (commits f2343653f5b, f003a7522bf). * I decided to initialize QueryDesc.planstate even in the cases where ExecInitNode() traversal is aborted in the middle on detecting CachedPlan invalidation such that it points to a partially initialized PlanState tree. My earlier thinking that each PlanState node need not be visited for resource cleanup in such cases was naive after all. To that end, I've fixed the ExecEndNode() subroutines of all Plan node types to account for potentially uninitialized fields. There are a couple of cases where I'm a bit doubtful though. In ExecEndCustomScan(), there's no indication in CustomScanState whether it's OK to call EndCustomScan() when BeginCustomScan() may not have been called. For ForeignScanState, I've assumed that ForeignScanState.fdw_state being set can be used as a marker that BeginForeignScan would have been called, though maybe that's not a solid assumption. I'm also attaching a new (small) patch 0003 that eliminates the loop-over-rangetable in ExecCloseRangeTableRelations() in favor of iterating over a new List field of EState named es_opened_relations, which is populated by ExecGetRangeTableRelation() with only the relations that were opened. This speeds up ExecCloseRangeTableRelations() significantly for the cases with many runtime-prunable partitions. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v35-0003-Track-opened-range-table-relations-in-a-List-in-.patch (2.3K, 2-v35-0003-Track-opened-range-table-relations-in-a-List-in-.patch) download | inline diff: From ff01e18a889f4e7ecb11d58488676395973656b0 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Mon, 13 Mar 2023 15:59:38 +0900 Subject: [PATCH v35 3/3] Track opened range table relations in a List in EState This makes ExecCloseRangeTableRelations faster when there are many relations in the range table but only a few are opened during execution, such as when run-time pruning kicks in on an Append containing 1000s of partition subplans. --- src/backend/executor/execMain.c | 9 +++++---- src/backend/executor/execUtils.c | 2 ++ src/include/nodes/execnodes.h | 2 ++ 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index fc0d2ca481..380739e3a2 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -1630,12 +1630,13 @@ ExecCloseResultRelations(EState *estate) void ExecCloseRangeTableRelations(EState *estate) { - int i; + ListCell *lc; - for (i = 0; i < estate->es_range_table_size; i++) + foreach(lc, estate->es_opened_relations) { - if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + Relation rel = lfirst(lc); + + table_close(rel, NoLock); } } diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index a485e7dfc5..f7053072d9 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -829,6 +829,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) } estate->es_relations[rti - 1] = rel; + estate->es_opened_relations = lappend(estate->es_opened_relations, + rel); } return rel; diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index c6b3885bf6..214a5f5ea4 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -617,6 +617,8 @@ typedef struct EState Index es_range_table_size; /* size of the range table arrays */ Relation *es_relations; /* Array of per-range-table-entry Relation * pointers, or NULL if not yet opened */ + List *es_opened_relations; /* List of non-NULL entries in + * es_relations in no specific order */ struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ -- 2.35.3 [application/octet-stream] v35-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch (130.2K, 3-v35-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 48465ed72fc40ac3f26c27703267513295b288f6 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v35 2/3] Move AcquireExecutorLocks()'s responsibility into the executor This commit introduces a new executor flag EXEC_FLAG_GET_LOCKS that should be passed in eflags to ExecutorStart() if the PlannedStmt comes from a CachedPlan. When set, the executor will take locks on any relations referenced in the plan nodes that need to be initialized for execution. That excludes any partitions that can be pruned during the executor initialization phase, that is, based on the values of only the external (PARAM_EXTERN) parameters. Relations that are not explicitly mentioned in the plan tree, such as views and non-leaf partition parents whose children are mentioned in Append/MergeAppend nodes, are locked separately. After taking each lock, the executor calls CachedPlanStillValid() to check if CachedPlan.is_valid has been reset by PlanCacheRelCallback() due to concurrent modification of relations referenced in the plan. If it is found that the CachedPlan is indeed invalid, the recursive ExecInitNode() traversal is aborted at that point. To allow the proper cleanup of such a partially initialized planstate tree, ExecEndNode() subroutines of various plan nodes have been fixed to account for potentially uninitialized fields. It is the caller's (of ExecutorStart()) responsibility to call ExecutorEnd() even on a QueryDesc containing such a partially initialized PlanState tree. Call sites that use plancache (GetCachedPlan) to get the plan trees to pass to the executor for execution should now be prepared to handle the case that the plan tree may be flagged by the executor as stale as described above. To that end, this commit refactors the relevant code sites to move the ExecutorStart() call closer to the GetCachedPlan() call to reduce the friction in the cases where replanning is needed due to a CachedPlan being marked stale in this manner. Callers must check that QueryDesc.plan_valid is true before passing it on to ExecutorRun() for execution. PortalStart() now performs CreateQueryDesc() and ExecutorStart() for all portal strategies, including those pertaining to multiple queries. The QueryDescs for strategies handled by PortalRunMulti() are remembered in the Portal in a new List field 'qdescs', allocated in a new memory context 'queryContext'. This new arrangment is to make it easier to discard and recreate a Portal if the CachedPlan goes stale during setup. --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 146 +++++--- src/backend/commands/extension.c | 2 + src/backend/commands/matview.c | 3 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 32 +- src/backend/executor/execMain.c | 89 ++++- src/backend/executor/execParallel.c | 8 +- src/backend/executor/execPartition.c | 4 + src/backend/executor/execProcnode.c | 9 + src/backend/executor/execUtils.c | 60 +++- src/backend/executor/functions.c | 2 + src/backend/executor/nodeAgg.c | 23 +- src/backend/executor/nodeAppend.c | 23 +- src/backend/executor/nodeBitmapAnd.c | 10 +- src/backend/executor/nodeBitmapHeapscan.c | 10 +- src/backend/executor/nodeBitmapIndexscan.c | 2 + src/backend/executor/nodeBitmapOr.c | 10 +- src/backend/executor/nodeCtescan.c | 6 +- src/backend/executor/nodeCustom.c | 12 +- src/backend/executor/nodeForeignscan.c | 22 +- src/backend/executor/nodeFunctionscan.c | 3 +- src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 5 +- src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 13 +- src/backend/executor/nodeIncrementalSort.c | 14 +- src/backend/executor/nodeIndexonlyscan.c | 7 +- src/backend/executor/nodeIndexscan.c | 7 +- src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 5 +- src/backend/executor/nodeMemoize.c | 12 +- src/backend/executor/nodeMergeAppend.c | 23 +- src/backend/executor/nodeMergejoin.c | 10 +- src/backend/executor/nodeModifyTable.c | 13 +- .../executor/nodeNamedtuplestorescan.c | 3 +- src/backend/executor/nodeNestloop.c | 7 +- src/backend/executor/nodeProjectSet.c | 5 +- src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 5 +- src/backend/executor/nodeSamplescan.c | 5 +- src/backend/executor/nodeSeqscan.c | 5 +- src/backend/executor/nodeSetOp.c | 5 +- src/backend/executor/nodeSort.c | 8 +- src/backend/executor/nodeSubqueryscan.c | 5 +- src/backend/executor/nodeTableFuncscan.c | 3 +- src/backend/executor/nodeTidrangescan.c | 5 +- src/backend/executor/nodeTidscan.c | 5 +- src/backend/executor/nodeUnique.c | 5 +- src/backend/executor/nodeValuesscan.c | 3 +- src/backend/executor/nodeWindowAgg.c | 55 +++- src/backend/executor/nodeWorktablescan.c | 3 +- src/backend/executor/spi.c | 53 ++- src/backend/nodes/outfuncs.c | 1 + src/backend/nodes/readfuncs.c | 1 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 13 +- src/backend/tcop/pquery.c | 311 ++++++++++-------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 ++------ src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 12 + src/include/nodes/execnodes.h | 2 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 14 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 117 +++++++ .../specs/cached-plan-replan.spec | 50 +++ 82 files changed, 1213 insertions(+), 422 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index beea1ac687..e9f77d5711 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index e57bda7b62..acae5b455b 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,93 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,29 +597,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - Assert(plannedstmt->commandType != CMD_UTILITY); - - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -546,38 +615,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4851,6 +4888,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index 02ff4a9a7f..2d2ef98b54 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -780,11 +780,13 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); ExecutorStart(qdesc, 0); + Assert(qdesc->plan_valid); ExecutorRun(qdesc, ForwardScanDirection, 0, true); ExecutorFinish(qdesc); ExecutorEnd(qdesc); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..9adaf6c527 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); /* call ExecutorStart to prepare the plan for execution */ ExecutorStart(queryDesc, 0); + Assert(queryDesc->plan_valid); /* run the plan */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..c9070ed97f 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, it + * must be recreated if portal->plan_valid is false which tells that the + * cached plan was found to have been invalidated when initializing one of + * the plan trees contained in it. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index b32f419176..fc0d2ca481 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,32 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan, in which case only + * some of the relations referenced in the plan would have been locked; to + * wit, those that AcquirePlannerLocks() deems necessary. Locks necessary + * to fully validate such a plan tree, including relations that are added by + * the planner, will taken when initializing the plan tree in InitPlan(); the + * the caller must have set the EXEC_FLAG_GET_LOCKS bit in eflags. If the + * CachedPlan gets invalidated as these locks are taken, plan tree + * initialization is suspended at the point when the invalidation is first + * detected and InitPlan() returns after setting queryDesc->plan_valid to + * false. queryDesc->planstate would be pointing to a potentially partially + * initialized PlanState tree in that case. Callers must retry the execution + * with a freshly created CachedPlan in that case, after properly freeing the + * partially valid QueryDesc. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + (CachedPlanStillValid(queryDesc->cplan) && + (eflags & EXEC_FLAG_GET_LOCKS) != 0)); + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +603,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +816,19 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } - /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -801,20 +839,32 @@ InitPlan(QueryDesc *queryDesc, int eflags) Plan *plan = plannedstmt->planTree; List *rangeTable = plannedstmt->rtable; EState *estate = queryDesc->estate; - PlanState *planstate; + PlanState *planstate = NULL; TupleDesc tupType; ListCell *l; int i; /* - * Do permissions checks + * Set up range table in EState. */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + + /* Make sure ExecPlanStillValid() can work. */ + estate->es_cachedplan = queryDesc->cplan; /* - * initialize the node's execution state + * Lock any views that were mentioned in the query if needed. View + * relations must be locked separately like this, because they are not + * referenced in the plan tree. */ - ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + ExecLockViewRelations(plannedstmt->viewRelations, estate); + if (!ExecPlanStillValid(estate)) + goto failed; + + /* + * Do permissions checks + */ + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); estate->es_plannedstmt = plannedstmt; estate->es_part_prune_infos = plannedstmt->partPruneInfos; @@ -849,6 +899,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -919,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); + if (!ExecPlanStillValid(estate)) + goto failed; i++; } @@ -929,6 +983,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -972,6 +1028,17 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such. Note that we do + * set planstate, even if it may only be partially initialized, so that + * ExecEndPlan() can process it. + */ + queryDesc->planstate = planstate; + queryDesc->plan_valid = false; } /* @@ -1389,7 +1456,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -2797,7 +2864,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2884,6 +2952,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } @@ -2937,6 +3006,10 @@ EvalPlanQualEnd(EPQState *epqstate) MemoryContext oldcontext; ListCell *l; + /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */ + if (epqstate->parentestate == NULL) + return; + rtsize = epqstate->parentestate->es_range_table_size; /* diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..df4cc5ddaf 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); @@ -1432,6 +1437,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc) /* Start up the executor */ queryDesc->plannedstmt->jitFlags = fpes->jit_flags; ExecutorStart(queryDesc, fpes->eflags); + Assert(queryDesc->plan_valid); /* Special executor initialization steps for parallel workers */ queryDesc->planstate->state->es_query_dsa = area; diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index fd6ca8a5d9..ae6a974e7a 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -1817,6 +1817,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1943,6 +1945,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..6f3c37b6fd 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return result; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -403,6 +406,12 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); subps = lappend(subps, sstate); + if (!ExecPlanStillValid(estate)) + { + /* Don't lose track of those initialized. */ + result->initPlan = subps; + return result; + } } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 012dbb6965..a485e7dfc5 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -804,7 +804,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -833,6 +834,61 @@ ExecGetRangeTableRelation(EState *estate, Index rti) return rel; } +/* + * ExecLockViewRelations + * Lock view relations, if any, in a given query + */ +void +ExecLockViewRelations(List *viewRelations, EState *estate) +{ + ListCell *lc; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + LockRelationOid(rte->relid, rte->rellockmode); + } +} + +/* + * ExecLockAppendNonLeafRelations + * Lock non-leaf relations whose children are scanned by a given + * Append/MergeAppend node + */ +void +ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids) +{ + ListCell *l; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(l, allpartrelids) + { + Bitmapset *partrelids = lfirst_node(Bitmapset, l); + int i; + + i = -1; + while ((i = bms_next_member(partrelids, i)) > 0) + { + RangeTblEntry *rte = exec_rt_fetch(i, estate); + + LockRelationOid(rte->relid, rte->rellockmode); + } + } +} + /* * ExecInitResultRelation * Open relation given by the passed-in RT index and fill its @@ -848,6 +904,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..f8c9de1fda 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, @@ -868,6 +869,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) else eflags = 0; /* default run-to-completion flags */ ExecutorStart(es->qd, eflags); + Assert(es->qd->plan_valid); } es->status = F_EXEC_RUN; diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 19342a420c..06e0d7d149 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3134,15 +3134,18 @@ hashagg_reset_spill_state(AggState *aggstate) { HashAggSpill *spill = &aggstate->hash_spills[setno]; - pfree(spill->ntuples); - pfree(spill->partitions); + if (spill->ntuples) + pfree(spill->ntuples); + if (spill->partitions) + pfree(spill->partitions); } pfree(aggstate->hash_spills); aggstate->hash_spills = NULL; } /* free batches */ - list_free_deep(aggstate->hash_batches); + if (aggstate->hash_batches) + list_free_deep(aggstate->hash_batches); aggstate->hash_batches = NIL; /* close tape set */ @@ -3296,6 +3299,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return aggstate; /* * initialize source tuple type. @@ -4336,10 +4341,13 @@ ExecEndAgg(AggState *node) { AggStatePerTrans pertrans = &node->pertrans[transno]; - for (setno = 0; setno < numGroupingSets; setno++) + if (pertrans) { - if (pertrans->sortstates[setno]) - tuplesort_end(pertrans->sortstates[setno]); + for (setno = 0; setno < numGroupingSets; setno++) + { + if (pertrans->sortstates[setno]) + tuplesort_end(pertrans->sortstates[setno]); + } } } @@ -4357,7 +4365,8 @@ ExecEndAgg(AggState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index c185b11c67..091f979c46 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -109,10 +109,11 @@ AppendState * ExecInitAppend(Append *node, EState *estate, int eflags) { AppendState *appendstate = makeNode(AppendState); - PlanState **appendplanstates; + PlanState **appendplanstates = NULL; Bitmapset *validsubplans; Bitmapset *asyncplans; int nplans; + int ninited = 0; int nasyncplans; int firstvalid; int i, @@ -133,6 +134,15 @@ ExecInitAppend(Append *node, EState *estate, int eflags) appendstate->as_syncdone = false; appendstate->as_begun = false; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -148,6 +158,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + goto early_exit; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -222,11 +234,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } appendstate->as_first_partial_plan = firstvalid; - appendstate->appendplans = appendplanstates; - appendstate->as_nplans = nplans; /* Initialize async state */ appendstate->as_asyncplans = asyncplans; @@ -276,6 +289,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags) /* For parallel query, this will be overridden later. */ appendstate->choose_next_subplan = choose_next_subplan_locally; +early_exit: + appendstate->appendplans = appendplanstates; + appendstate->as_nplans = ninited; + return appendstate; } diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..acc6c50e20 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -57,6 +57,7 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) BitmapAndState *bitmapandstate = makeNode(BitmapAndState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -77,8 +78,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) bitmapandstate->ps.plan = (Plan *) node; bitmapandstate->ps.state = estate; bitmapandstate->ps.ExecProcNode = ExecBitmapAnd; - bitmapandstate->bitmapplans = bitmapplanstates; - bitmapandstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -89,6 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -99,6 +101,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmapandstate->bitmapplans = bitmapplanstates; + bitmapandstate->nplans = ninited; + return bitmapandstate; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..e6a689eefb 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -665,7 +665,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subplans @@ -693,7 +694,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) /* * close heap scan */ - table_endscan(scanDesc); + if (scanDesc) + table_endscan(scanDesc); } /* ---------------------------------------------------------------- @@ -763,11 +765,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c index 83ec9ede89..cc8332ef68 100644 --- a/src/backend/executor/nodeBitmapIndexscan.c +++ b/src/backend/executor/nodeBitmapIndexscan.c @@ -263,6 +263,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->biss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..babad1b4b2 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -58,6 +58,7 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) BitmapOrState *bitmaporstate = makeNode(BitmapOrState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -78,8 +79,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) bitmaporstate->ps.plan = (Plan *) node; bitmaporstate->ps.state = estate; bitmaporstate->ps.ExecProcNode = ExecBitmapOr; - bitmaporstate->bitmapplans = bitmapplanstates; - bitmaporstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -90,6 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -100,6 +102,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmaporstate->bitmapplans = bitmapplanstates; + bitmaporstate->nplans = ninited; + return bitmaporstate; } diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c index cc4c4243e2..eed5b75a4f 100644 --- a/src/backend/executor/nodeCtescan.c +++ b/src/backend/executor/nodeCtescan.c @@ -297,14 +297,16 @@ ExecEndCteScan(CteScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * If I am the leader, free the tuplestore. */ if (node->leader == node) { - tuplestore_end(node->cte_table); + if (node->cte_table) + tuplestore_end(node->cte_table); node->cte_table = NULL; } } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..b03499fae5 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return css; css->ss.ss_currentRelation = scan_rel; } @@ -127,6 +129,10 @@ ExecCustomScan(PlanState *pstate) void ExecEndCustomScan(CustomScanState *node) { + /* + * XXX - BeginCustomScan() may not have occurred if ExecInitCustomScan() + * hit the early exit case. + */ Assert(node->methods->EndCustomScan != NULL); node->methods->EndCustomScan(node); @@ -134,8 +140,10 @@ ExecEndCustomScan(CustomScanState *node) ExecFreeExprContext(&node->ss.ps); /* Clean out the tuple table */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } void diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..d3f0a65485 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * Tell the FDW to initialize the scan. @@ -300,14 +304,17 @@ ExecEndForeignScan(ForeignScanState *node) ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; EState *estate = node->ss.ps.state; - /* Let the FDW shut down */ - if (plan->operation != CMD_SELECT) + /* Let the FDW shut down if needed. */ + if (node->fdw_state) { - if (estate->es_epq_active == NULL) - node->fdwroutine->EndDirectModify(node); + if (plan->operation != CMD_SELECT) + { + if (estate->es_epq_active == NULL) + node->fdwroutine->EndDirectModify(node); + } + else + node->fdwroutine->EndForeignScan(node); } - else - node->fdwroutine->EndForeignScan(node); /* Shut down any outer plan. */ if (outerPlanState(node)) @@ -319,7 +326,8 @@ ExecEndForeignScan(ForeignScanState *node) /* clean out the tuple table */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c index dd06ef8aee..792ecda4a9 100644 --- a/src/backend/executor/nodeFunctionscan.c +++ b/src/backend/executor/nodeFunctionscan.c @@ -533,7 +533,8 @@ ExecEndFunctionScan(FunctionScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release slots and tuplestore resources diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..365d3af3e4 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gatherstate; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..8d2809f079 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gm_state; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..e0832bb778 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return grpstate; /* * Initialize scan slot and type. @@ -231,7 +233,8 @@ ExecEndGroup(GroupState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index eceee99374..6afc04edf1 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -379,6 +379,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return hashstate; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index b215e3f59a..9efb238d1c 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -659,8 +659,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* @@ -781,9 +785,12 @@ ExecEndHashJoin(HashJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->hj_OuterTupleSlot); - ExecClearTuple(node->hj_HashTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->hj_OuterTupleSlot) + ExecClearTuple(node->hj_OuterTupleSlot); + if (node->hj_HashTupleSlot) + ExecClearTuple(node->hj_HashTupleSlot); /* * clean up subtrees diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..6b2da56044 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return incrsortstate; /* * Initialize scan slot and type. @@ -1080,12 +1082,16 @@ ExecEndIncrementalSort(IncrementalSortState *node) SO_printf("ExecEndIncrementalSort: shutting down sort node\n"); /* clean out the scan tuple */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* must drop standalone tuple slots from outer node */ - ExecDropSingleTupleTableSlot(node->group_pivot); - ExecDropSingleTupleTableSlot(node->transfer_tuple); + if (node->group_pivot) + ExecDropSingleTupleTableSlot(node->group_pivot); + if (node->transfer_tuple) + ExecDropSingleTupleTableSlot(node->transfer_tuple); /* * Release tuplesort resources. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..b60a086464 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -394,7 +394,8 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if(node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -565,6 +568,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..628c233919 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -808,7 +808,8 @@ ExecEndIndexScan(IndexScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -970,6 +973,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->iss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..2fcbde74ed 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return limitstate; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..3a8aa2b5a4 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return lrstate; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..f146ebb1d7 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return matstate; /* * Initialize result type and slot. No need to initialize projection info @@ -242,7 +244,8 @@ ExecEndMaterial(MaterialState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 74f7d21bc8..a6df43ba19 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -931,6 +931,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mstate; /* * Initialize return slot and type. No need to initialize projection info @@ -1036,6 +1038,7 @@ ExecEndMemoize(MemoizeState *node) { #ifdef USE_ASSERT_CHECKING /* Validate the memory accounting code is correct in assert builds. */ + if (node->hashtable) { int count; uint64 mem = 0; @@ -1082,11 +1085,14 @@ ExecEndMemoize(MemoizeState *node) } /* Remove the cache context */ - MemoryContextDelete(node->tableContext); + if (node->tableContext) + MemoryContextDelete(node->tableContext); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to cache result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * free exprcontext diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..40bba35499 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -65,9 +65,10 @@ MergeAppendState * ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) { MergeAppendState *mergestate = makeNode(MergeAppendState); - PlanState **mergeplanstates; + PlanState **mergeplanstates = NULL; Bitmapset *validsubplans; int nplans; + int ninited = 0; int i, j; @@ -81,6 +82,15 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) mergestate->ps.state = estate; mergestate->ps.ExecProcNode = ExecMergeAppend; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -96,6 +106,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return mergestate; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -122,8 +134,6 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) } mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *)); - mergestate->mergeplans = mergeplanstates; - mergestate->ms_nplans = nplans; mergestate->ms_slots = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans); mergestate->ms_heap = binaryheap_allocate(nplans, heap_compare_slots, @@ -152,6 +162,9 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } mergestate->ps.ps_ProjInfo = NULL; @@ -188,6 +201,10 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) */ mergestate->ms_initialized = false; +early_exit: + mergestate->mergeplans = mergeplanstates; + mergestate->ms_nplans = ninited; + return mergestate; } diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..968be05568 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return mergestate; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* @@ -1642,8 +1646,10 @@ ExecEndMergeJoin(MergeJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->mj_MarkedTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->mj_MarkedTupleSlot) + ExecClearTuple(node->mj_MarkedTupleSlot); /* * shut down the subplans diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 3fa2b930a5..7cdbe7f5f5 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -3919,6 +3919,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) Plan *subplan = outerPlan(node); CmdType operation = node->operation; int nrels = list_length(node->resultRelations); + int ninited = 0; ResultRelInfo *resultRelInfo; List *arowmarks; ListCell *l; @@ -3940,7 +3941,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) mtstate->canSetTag = node->canSetTag; mtstate->mt_done = false; - mtstate->mt_nrels = nrels; mtstate->resultRelInfo = (ResultRelInfo *) palloc(nrels * sizeof(ResultRelInfo)); @@ -3975,6 +3975,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -4001,6 +4004,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * For child result relations, store the root result relation @@ -4028,11 +4033,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * Do additional per-result-relation initialization. */ - for (i = 0; i < nrels; i++) + for (i = 0; i < nrels; i++, ninited++) { resultRelInfo = &mtstate->resultRelInfo[i]; @@ -4381,6 +4388,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) estate->es_auxmodifytables = lcons(mtstate, estate->es_auxmodifytables); +early_exit: + mtstate->mt_nrels = ninited; return mtstate; } diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c index 46832ad82f..1f92c43d3b 100644 --- a/src/backend/executor/nodeNamedtuplestorescan.c +++ b/src/backend/executor/nodeNamedtuplestorescan.c @@ -174,7 +174,8 @@ ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..deda0c2559 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; /* * Initialize result slot, type and projection. @@ -372,7 +376,8 @@ ExecEndNestLoop(NestLoopState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); /* * close down subplans diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..85d20c4680 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return state; /* * we don't use inner plan @@ -328,7 +330,8 @@ ExecEndProjectSet(ProjectSetState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..967fe4f287 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..c549b684a3 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return resstate; /* * we don't use inner plan @@ -248,7 +250,8 @@ ExecEndResult(ResultState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..b3bc9b1f77 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; @@ -198,7 +200,8 @@ ExecEndSampleScan(SampleScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..e7ca19ee4e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, @@ -200,7 +202,8 @@ ExecEndSeqScan(SeqScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..95950a5c20 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return setopstate; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* @@ -583,7 +585,8 @@ void ExecEndSetOp(SetOpState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* free subsidiary stuff including hashtable */ if (node->tableContext) diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..89fef86aba 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return sortstate; /* * Initialize scan slot and type. @@ -306,9 +308,11 @@ ExecEndSort(SortState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * Release tuplesort resources diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..9b8cddc89f 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return subquerystate; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) @@ -177,7 +179,8 @@ ExecEndSubqueryScan(SubqueryScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subquery diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c index 0c6c912778..d7536953f1 100644 --- a/src/backend/executor/nodeTableFuncscan.c +++ b/src/backend/executor/nodeTableFuncscan.c @@ -223,7 +223,8 @@ ExecEndTableFuncScan(TableFuncScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..1ae451d7a6 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -342,7 +342,8 @@ ExecEndTidRangeScan(TidRangeScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -386,6 +387,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidrangestate; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 862bd0330b..9fe76b1c60 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -483,7 +483,8 @@ ExecEndTidScan(TidScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -529,6 +530,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidstate; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..69f23b02c6 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return uniquestate; /* * Initialize result slot and type. Unique nodes do no projections, so @@ -169,7 +171,8 @@ void ExecEndUnique(UniqueState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); ExecFreeExprContext(&node->ps); diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c index 32ace63017..f5dedbab63 100644 --- a/src/backend/executor/nodeValuesscan.c +++ b/src/backend/executor/nodeValuesscan.c @@ -340,7 +340,8 @@ ExecEndValuesScan(ValuesScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 7c07fb0684..616bb97675 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -1334,7 +1334,7 @@ release_partition(WindowAggState *winstate) WindowStatePerFunc perfuncstate = &(winstate->perfunc[i]); /* Release any partition-local state of this window function */ - if (perfuncstate->winobj) + if (perfuncstate && perfuncstate->winobj) perfuncstate->winobj->localmem = NULL; } @@ -1344,12 +1344,17 @@ release_partition(WindowAggState *winstate) * any aggregate temp data). We don't rely on retail pfree because some * aggregates might have allocated data we don't have direct pointers to. */ - MemoryContextResetAndDeleteChildren(winstate->partcontext); - MemoryContextResetAndDeleteChildren(winstate->aggcontext); - for (i = 0; i < winstate->numaggs; i++) + if (winstate->partcontext) + MemoryContextResetAndDeleteChildren(winstate->partcontext); + if (winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->aggcontext); + if (winstate->peragg) { - if (winstate->peragg[i].aggcontext != winstate->aggcontext) - MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + for (i = 0; i < winstate->numaggs; i++) + { + if (winstate->peragg[i].aggcontext != winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + } } if (winstate->buffer) @@ -2451,6 +2456,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return winstate; /* * initialize source tuple type (which is also the tuple type that we'll @@ -2679,11 +2686,16 @@ ExecEndWindowAgg(WindowAggState *node) release_partition(node); - ExecClearTuple(node->ss.ss_ScanTupleSlot); - ExecClearTuple(node->first_part_slot); - ExecClearTuple(node->agg_row_slot); - ExecClearTuple(node->temp_slot_1); - ExecClearTuple(node->temp_slot_2); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->first_part_slot) + ExecClearTuple(node->first_part_slot); + if (node->agg_row_slot) + ExecClearTuple(node->agg_row_slot); + if (node->temp_slot_1) + ExecClearTuple(node->temp_slot_1); + if (node->temp_slot_2) + ExecClearTuple(node->temp_slot_2); if (node->framehead_slot) ExecClearTuple(node->framehead_slot); if (node->frametail_slot) @@ -2696,16 +2708,23 @@ ExecEndWindowAgg(WindowAggState *node) node->ss.ps.ps_ExprContext = node->tmpcontext; ExecFreeExprContext(&node->ss.ps); - for (i = 0; i < node->numaggs; i++) + if (node->peragg) { - if (node->peragg[i].aggcontext != node->aggcontext) - MemoryContextDelete(node->peragg[i].aggcontext); + for (i = 0; i < node->numaggs; i++) + { + if (node->peragg[i].aggcontext != node->aggcontext) + MemoryContextDelete(node->peragg[i].aggcontext); + } } - MemoryContextDelete(node->partcontext); - MemoryContextDelete(node->aggcontext); + if (node->partcontext) + MemoryContextDelete(node->partcontext); + if (node->aggcontext) + MemoryContextDelete(node->aggcontext); - pfree(node->perfunc); - pfree(node->peragg); + if (node->perfunc) + pfree(node->perfunc); + if (node->peragg) + pfree(node->peragg); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c index 0c13448236..d70c6afde3 100644 --- a/src/backend/executor/nodeWorktablescan.c +++ b/src/backend/executor/nodeWorktablescan.c @@ -200,7 +200,8 @@ ExecEndWorkTableScan(WorkTableScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index e3a170c38b..26a9ea342a 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2668,14 +2680,36 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + + /* Take locks if using a CachedPlan */ + if (qdesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + Assert(cplan); + ReleaseCachedPlan(cplan, plan_owner); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2850,10 +2884,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2897,14 +2930,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index ba00b99249..955286513d 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -513,6 +513,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ WRITE_OID_FIELD(relid); + WRITE_CHAR_FIELD(relkind); WRITE_INT_FIELD(rellockmode); WRITE_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index f3629cdfd1..3bc5a6dca0 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -480,6 +480,7 @@ _readRangeTblEntry(void) READ_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ READ_OID_FIELD(relid); + READ_CHAR_FIELD(relkind); READ_INT_FIELD(rellockmode); READ_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 62b3ec96cc..5f3ffd98af 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 5cc8366af6..f13240bf33 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -604,6 +605,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index 980dc1816f..1631c8b993 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1849,11 +1849,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..6d0ea07801 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1199,6 +1199,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1703,6 +1704,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1994,10 +1996,19 @@ exec_bind_message(StringInfo input_message) PopActiveSnapshot(); /* - * And we're ready to start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..c93a950d7f 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt, qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +353,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +419,56 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti(). */ + if (portal->strategy == PORTAL_ONE_RETURNING || + portal->strategy == PORTAL_ONE_MOD_WITH) + portal->qdescs = list_make1(queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +476,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +500,90 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool first = true; + + /* Take locks if using a CachedPlan */ + myeflags = 0; + if (portal->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* + * Push the snapshot to be used by the executor. + */ + if (!is_utility) + { + /* + * Must copy the snapshot if we'll need to update + * its command ID. + */ + if (!first) + PushCopiedSnapshot(GetTransactionSnapshot()); + else + PushActiveSnapshot(GetTransactionSnapshot()); + } + + /* + * From the 2nd statement onwards, update the command + * ID and the snapshot to match. + */ + if (!first) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + first = false; + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + !is_utility ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + if (is_utility) + continue; + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + ExecutorStart(queryDesc, myeflags); + PopActiveSnapshot(); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + portal->plan_valid = false; + goto early_exit; + } + } + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +595,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1193,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1214,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1271,23 +1272,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1362,15 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c07382051d..38ae43e24b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..da39b2e4ff 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +107,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index 946abc0051..5f860662b1 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -59,6 +60,8 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should the executor lock + * relations? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -245,6 +248,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode @@ -579,6 +589,8 @@ exec_rt_fetch(Index rti, EState *estate) } extern Relation ExecGetRangeTableRelation(EState *estate, Index rti); +extern void ExecLockViewRelations(List *viewRelations, EState *estate); +extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids); extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Index rti); diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index bc67cb9ed8..c6b3885bf6 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -621,6 +621,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d61a62da19..9b888b0d75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index a0bb16cff4..7cae624bbd 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..8990fe72e3 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor on every relation lock taken when initializing the + * plan tree in the CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..4f450b9d9b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,117 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +-------------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo11 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo11_a_idx + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +----------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo11 foo_1 + Filter: (a = $1) +(4 rows) + + +starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock +step s1prep2: SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Index Only Scan using foo11_a_idx on foo11 t1 + -> Materialize + -> Index Scan using foo11_a_idx on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(18 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec2: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec2: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +--------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Sort + Sort Key: t1.a + -> Seq Scan on foo11 t1 + -> Sort + Sort Key: t2.a + -> Seq Scan on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(21 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..67cfed7044 --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,50 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a); + CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1); + CREATE INDEX foo11_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +step "s1prep2" { SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +step "s1exec2" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo11_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" +permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock" -- 2.35.3 [application/octet-stream] v35-0001-Add-field-to-store-partitioned-relids-to-Append-.patch (20.2K, 4-v35-0001-Add-field-to-store-partitioned-relids-to-Append-.patch) download | inline diff: From d4c6eddd02d048a42a9941ab7dc8dcf1411fef78 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Thu, 9 Mar 2023 11:26:06 +0900 Subject: [PATCH v35 1/3] Add field to store partitioned relids to Append/MergeAppend A future commit would like to move the timing of locking relations referenced in a cached plan to ExecInitNode() traversal of the plan tree from the current loop-over-rangetable in AcquireExecutorLocks(). Given that partitioned tables (their RT indexes) would not be accessible via the new way of finding the relations to lock, add a field to Append/MergeAppend to track them separately. This refactors the code to look up partitioned parent relids from a given list of leaf partition subpaths of an Append/MergeAppend out of make_partition_pruneinfo() into its own function called add_append_subpath_partrelids(). Though, the code needs to be generalized to the cases where child rels can be joinrels or upper (grouping) rels. Also, to make it easier to traverse the parent chain of a child grouping rel, this makes its RelOptInfo.parent to be set, which is already done for baserels and joinrels. --- src/backend/optimizer/plan/createplan.c | 36 +++++-- src/backend/optimizer/plan/planner.c | 3 + src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++ src/backend/partitioning/partprune.c | 124 +++------------------- src/include/nodes/plannodes.h | 14 +++ src/include/optimizer/appendinfo.h | 3 + src/include/partitioning/partprune.h | 3 +- 7 files changed, 194 insertions(+), 123 deletions(-) diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index fa09a6103b..94fe0a28ad 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -25,6 +25,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/appendinfo.h" #include "optimizer/clauses.h" #include "optimizer/cost.h" #include "optimizer/optimizer.h" @@ -1209,6 +1210,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) Oid *nodeCollations = NULL; bool *nodeNullsFirst = NULL; bool consider_async = false; + List *allpartrelids = NIL; /* * The subpaths list could be empty, if every child was proven empty by @@ -1350,18 +1352,24 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) ++nasyncplans; } + /* Populate partitioned parent relids. */ + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + plan->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ plan->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1381,7 +1389,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) if (prunequal != NIL) plan->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } plan->appendplans = subplans; @@ -1425,6 +1434,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, List *subplans = NIL; ListCell *subpaths; RelOptInfo *rel = best_path->path.parent; + List *allpartrelids = NIL; /* * We don't have the actual creation of the MergeAppend node split out @@ -1514,18 +1524,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, subplan = (Plan *) sort; } + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + node->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ node->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1545,7 +1560,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, if (prunequal != NIL) node->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } node->mergeplans = subplans; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index a1873ce26d..62b3ec96cc 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -7801,8 +7801,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root, agg_costs, gd, &child_extra, &child_partially_grouped_rel); + /* Mark as child of grouped_rel. */ + child_grouped_rel->parent = grouped_rel; if (child_partially_grouped_rel) { + child_partially_grouped_rel->parent = grouped_rel; partially_grouped_live_children = lappend(partially_grouped_live_children, child_partially_grouped_rel); diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c index 9d377385f1..4876742ab2 100644 --- a/src/backend/optimizer/util/appendinfo.c +++ b/src/backend/optimizer/util/appendinfo.c @@ -40,6 +40,7 @@ static void make_inh_translation_list(Relation oldrelation, AppendRelInfo *appinfo); static Node *adjust_appendrel_attrs_mutator(Node *node, adjust_appendrel_attrs_context *context); +static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); /* @@ -1031,3 +1032,136 @@ distribute_row_identity_vars(PlannerInfo *root) } } } + +/* + * add_append_subpath_partrelids + * Look up a child subpath's rel's partitioned parent relids up to + * parentrel and add the bitmapset containing those into + * 'allpartrelids' + */ +List * +add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids) +{ + RelOptInfo *prel = subpath->parent; + Relids partrelids = NULL; + + /* Nothing to do if there's no parent to begin with. */ + if (!IS_OTHER_REL(prel)) + return allpartrelids; + + /* + * Traverse up to the pathrel's topmost partitioned parent, collecting + * parent relids as we go; but stop if we reach parentrel. (Normally, a + * pathrel's topmost partitioned parent is either parentrel or a UNION ALL + * appendrel child of parentrel. But when handling partitionwise joins of + * multi-level partitioning trees, we can see an append path whose + * parentrel is an intermediate partitioned table.) + */ + do + { + Relids parent_relids = NULL; + + /* + * For simple child rels, we can simply get the parent relid from + * prel->parent. But for partitionwise join and aggregate child rels, + * while we can use prel->parent to move up the tree, parent relids to + * add into 'partrelids' must be found the hard way through the + * AppendInfoInfos, because 1) a joinrel's relids may point to RTE_JOIN + * entries, 2) topmost parent grouping rel's relids field is left NULL. + */ + if (IS_SIMPLE_REL(prel)) + { + prel = prel->parent; + /* Stop once we reach the root partitioned rel. */ + if (!IS_PARTITIONED_REL(prel)) + break; + parent_relids = bms_add_members(parent_relids, prel->relids); + } + else + { + AppendRelInfo **appinfos; + int nappinfos, + i; + + appinfos = find_appinfos_by_relids(root, prel->relids, + &nappinfos); + for (i = 0; i < nappinfos; i++) + { + AppendRelInfo *appinfo = appinfos[i]; + + parent_relids = bms_add_member(parent_relids, + appinfo->parent_relid); + } + pfree(appinfos); + prel = prel->parent; + } + /* accept this level as an interesting parent */ + partrelids = bms_add_members(partrelids, parent_relids); + if (prel == parentrel) + break; /* don't traverse above parentrel */ + } while (IS_OTHER_REL(prel)); + + if (partrelids == NULL) + return allpartrelids; + + return add_part_relids(allpartrelids, partrelids); +} + +/* + * add_part_relids + * Add new info to a list of Bitmapsets of partitioned relids. + * + * Within 'allpartrelids', there is one Bitmapset for each topmost parent + * partitioned rel. Each Bitmapset contains the RT indexes of the topmost + * parent as well as its relevant non-leaf child partitions. Since (by + * construction of the rangetable list) parent partitions must have lower + * RT indexes than their children, we can distinguish the topmost parent + * as being the lowest set bit in the Bitmapset. + * + * 'partrelids' contains the RT indexes of a parent partitioned rel, and + * possibly some non-leaf children, that are newly identified as parents of + * some subpath rel passed to make_partition_pruneinfo(). These are added + * to an appropriate member of 'allpartrelids'. + * + * Note that the list contains only RT indexes of partitioned tables that + * are parents of some scan-level relation appearing in the 'subpaths' that + * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are + * not allowed to be higher than the 'parentrel' associated with the append + * path. In this way, we avoid expending cycles on partitioned rels that + * can't contribute useful pruning information for the problem at hand. + * (It is possible for 'parentrel' to be a child partitioned table, and it + * is also possible for scan-level relations to be child partitioned tables + * rather than leaf partitions. Hence we must construct this relation set + * with reference to the particular append path we're dealing with, rather + * than looking at the full partitioning structure represented in the + * RelOptInfos.) + */ +static List * +add_part_relids(List *allpartrelids, Bitmapset *partrelids) +{ + Index targetpart; + ListCell *lc; + + /* We can easily get the lowest set bit this way: */ + targetpart = bms_next_member(partrelids, -1); + Assert(targetpart > 0); + + /* Look for a matching topmost parent */ + foreach(lc, allpartrelids) + { + Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); + Index currtarget = bms_next_member(currpartrelids, -1); + + if (targetpart == currtarget) + { + /* Found a match, so add any new RT indexes to this hierarchy */ + currpartrelids = bms_add_members(currpartrelids, partrelids); + lfirst(lc) = currpartrelids; + return allpartrelids; + } + } + /* No match, so add the new partition hierarchy to the list */ + return lappend(allpartrelids, partrelids); +} diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index 510145e3c0..3557e07082 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@ -138,7 +138,6 @@ typedef struct PruneStepResult } PruneStepResult; -static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); static List *make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *prunequal, @@ -221,33 +220,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context, * of scan paths for its child rels. * 'prunequal' is a list of potential pruning quals (i.e., restriction * clauses that are applicable to the appendrel). + * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents + * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every + * partition tree involved. */ int make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *subpaths, - List *prunequal) + List *prunequal, + List *allpartrelids) { PartitionPruneInfo *pruneinfo; Bitmapset *allmatchedsubplans = NULL; - List *allpartrelids; List *prunerelinfos; int *relid_subplan_map; ListCell *lc; int i; + Assert(list_length(allpartrelids) > 0); + /* - * Scan the subpaths to see which ones are scans of partition child - * relations, and identify their parent partitioned rels. (Note: we must - * restrict the parent partitioned rels to be parentrel or children of - * parentrel, otherwise we couldn't translate prunequal to match.) - * - * Also construct a temporary array to map from partition-child-relation - * relid to the index in 'subpaths' of the scan plan for that partition. + * Construct a temporary array to map from partition-child-relation relid + * to the index in 'subpaths' of the scan plan for that partition. * (Use of "subplan" rather than "subpath" is a bit of a misnomer, but * we'll let it stand.) For convenience, we use 1-based indexes here, so * that zero can represent an un-filled array entry. */ - allpartrelids = NIL; relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size); i = 1; @@ -256,50 +254,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, Path *path = (Path *) lfirst(lc); RelOptInfo *pathrel = path->parent; - /* We don't consider partitioned joins here */ - if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL) - { - RelOptInfo *prel = pathrel; - Bitmapset *partrelids = NULL; - - /* - * Traverse up to the pathrel's topmost partitioned parent, - * collecting parent relids as we go; but stop if we reach - * parentrel. (Normally, a pathrel's topmost partitioned parent - * is either parentrel or a UNION ALL appendrel child of - * parentrel. But when handling partitionwise joins of - * multi-level partitioning trees, we can see an append path whose - * parentrel is an intermediate partitioned table.) - */ - do - { - AppendRelInfo *appinfo; - - Assert(prel->relid < root->simple_rel_array_size); - appinfo = root->append_rel_array[prel->relid]; - prel = find_base_rel(root, appinfo->parent_relid); - if (!IS_PARTITIONED_REL(prel)) - break; /* reached a non-partitioned parent */ - /* accept this level as an interesting parent */ - partrelids = bms_add_member(partrelids, prel->relid); - if (prel == parentrel) - break; /* don't traverse above parentrel */ - } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL); - - if (partrelids) - { - /* - * Found some relevant parent partitions, which may or may not - * overlap with partition trees we already found. Add new - * information to the allpartrelids list. - */ - allpartrelids = add_part_relids(allpartrelids, partrelids); - /* Also record the subplan in relid_subplan_map[] */ - /* No duplicates please */ - Assert(relid_subplan_map[pathrel->relid] == 0); - relid_subplan_map[pathrel->relid] = i; - } - } + /* No duplicates please */ + Assert(relid_subplan_map[pathrel->relid] == 0); + relid_subplan_map[pathrel->relid] = i; i++; } @@ -368,63 +325,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, return list_length(root->partPruneInfos) - 1; } -/* - * add_part_relids - * Add new info to a list of Bitmapsets of partitioned relids. - * - * Within 'allpartrelids', there is one Bitmapset for each topmost parent - * partitioned rel. Each Bitmapset contains the RT indexes of the topmost - * parent as well as its relevant non-leaf child partitions. Since (by - * construction of the rangetable list) parent partitions must have lower - * RT indexes than their children, we can distinguish the topmost parent - * as being the lowest set bit in the Bitmapset. - * - * 'partrelids' contains the RT indexes of a parent partitioned rel, and - * possibly some non-leaf children, that are newly identified as parents of - * some subpath rel passed to make_partition_pruneinfo(). These are added - * to an appropriate member of 'allpartrelids'. - * - * Note that the list contains only RT indexes of partitioned tables that - * are parents of some scan-level relation appearing in the 'subpaths' that - * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are - * not allowed to be higher than the 'parentrel' associated with the append - * path. In this way, we avoid expending cycles on partitioned rels that - * can't contribute useful pruning information for the problem at hand. - * (It is possible for 'parentrel' to be a child partitioned table, and it - * is also possible for scan-level relations to be child partitioned tables - * rather than leaf partitions. Hence we must construct this relation set - * with reference to the particular append path we're dealing with, rather - * than looking at the full partitioning structure represented in the - * RelOptInfos.) - */ -static List * -add_part_relids(List *allpartrelids, Bitmapset *partrelids) -{ - Index targetpart; - ListCell *lc; - - /* We can easily get the lowest set bit this way: */ - targetpart = bms_next_member(partrelids, -1); - Assert(targetpart > 0); - - /* Look for a matching topmost parent */ - foreach(lc, allpartrelids) - { - Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); - Index currtarget = bms_next_member(currpartrelids, -1); - - if (targetpart == currtarget) - { - /* Found a match, so add any new RT indexes to this hierarchy */ - currpartrelids = bms_add_members(currpartrelids, partrelids); - lfirst(lc) = currpartrelids; - return allpartrelids; - } - } - /* No match, so add the new partition hierarchy to the list */ - return lappend(allpartrelids, partrelids); -} - /* * make_partitionedrel_pruneinfo * Build a List of PartitionedRelPruneInfos, one for each interesting diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 659bd05c0c..a0bb16cff4 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -270,6 +270,13 @@ typedef struct Append List *appendplans; int nasyncplans; /* # of asynchronous plans */ + /* + * RTIs of all partitioned tables whose children are scanned by + * appendplans. The list contains a bitmapset for every partition tree + * covered by this Append. + */ + List *allpartrelids; + /* * All 'appendplans' preceding this index are non-partial plans. All * 'appendplans' from this index onwards are partial plans. @@ -294,6 +301,13 @@ typedef struct MergeAppend List *mergeplans; + /* + * RTIs of all partitioned tables whose children are scanned by + * mergeplans. The list contains a bitmapset for every partition tree + * covered by this MergeAppend. + */ + List *allpartrelids; + /* these fields are just like the sort-key info in struct Sort: */ /* number of sort-key columns */ diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h index a05f91f77d..1621a7319a 100644 --- a/src/include/optimizer/appendinfo.h +++ b/src/include/optimizer/appendinfo.h @@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex, RangeTblEntry *target_rte, Relation target_relation); extern void distribute_row_identity_vars(PlannerInfo *root); +extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids); #endif /* APPENDINFO_H */ diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h index c0d6889d47..2d907d31d4 100644 --- a/src/include/partitioning/partprune.h +++ b/src/include/partitioning/partprune.h @@ -73,7 +73,8 @@ typedef struct PartitionPruneContext extern int make_partition_pruneinfo(struct PlannerInfo *root, struct RelOptInfo *parentrel, List *subpaths, - List *prunequal); + List *prunequal, + List *allpartrelids); extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel); extern Bitmapset *get_matching_partitions(PartitionPruneContext *context, List *pruning_steps); -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-03-22 12:48 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-03-22 12:48 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> On Tue, Mar 14, 2023 at 7:07 PM Amit Langote <[email protected]> wrote: > On Thu, Mar 2, 2023 at 10:52 PM Amit Langote <[email protected]> wrote: > > I think I have figured out what might be going wrong on that cfbot > > animal after building with the same CPPFLAGS as that animal locally. > > I had forgotten to update _out/_readRangeTblEntry() to account for the > > patch's change that a view's RTE_SUBQUERY now also preserves relkind > > in addition to relid and rellockmode for the locking consideration. > > > > Also, I noticed that a multi-query Portal execution with rules was > > failing (thanks to a regression test added in a7d71c41db) because of > > the snapshot used for the 2nd query onward not being updated for > > command ID change under patched model of multi-query Portal execution. > > To wit, under the patched model, all queries in the multi-query Portal > > case undergo ExecutorStart() before any of it is run with > > ExecutorRun(). The patch hadn't changed things however to update the > > snapshot's command ID for the 2nd query onwards, which caused the > > aforementioned test case to fail. > > > > This new model does however mean that the 2nd query onwards must use > > PushCopiedSnapshot() given the current requirement of > > UpdateActiveSnapshotCommandId() that the snapshot passed to it must > > not be referenced anywhere else. The new model basically requires > > that each query's QueryDesc points to its own copy of the > > ActiveSnapshot. That may not be a thing in favor of the patched model > > though. For now, I haven't been able to come up with a better > > alternative. > > Here's a new version addressing the following 2 points. > > * Like views, I realized that non-leaf relations of partition trees > scanned by an Append/MergeAppend would need to be locked separately, > because ExecInitNode() traversal of the plan tree would not account > for them. That is, they are not opened using > ExecGetRangeTableRelation() or ExecOpenScanRelation(). One exception > is that some (if not all) of those non-leaf relations may be > referenced in PartitionPruneInfo and so locked as part of initializing > the corresponding PartitionPruneState, but I decided not to complicate > the code to filter out such relations from the set locked separately. > To carry the set of relations to lock, the refactoring patch 0001 > re-introduces the List of Bitmapset field named allpartrelids into > Append/MergeAppend nodes, which we had previously removed on the > grounds that those relations need not be locked separately (commits > f2343653f5b, f003a7522bf). > > * I decided to initialize QueryDesc.planstate even in the cases where > ExecInitNode() traversal is aborted in the middle on detecting > CachedPlan invalidation such that it points to a partially initialized > PlanState tree. My earlier thinking that each PlanState node need not > be visited for resource cleanup in such cases was naive after all. To > that end, I've fixed the ExecEndNode() subroutines of all Plan node > types to account for potentially uninitialized fields. There are a > couple of cases where I'm a bit doubtful though. In > ExecEndCustomScan(), there's no indication in CustomScanState whether > it's OK to call EndCustomScan() when BeginCustomScan() may not have > been called. For ForeignScanState, I've assumed that > ForeignScanState.fdw_state being set can be used as a marker that > BeginForeignScan would have been called, though maybe that's not a > solid assumption. > > I'm also attaching a new (small) patch 0003 that eliminates the > loop-over-rangetable in ExecCloseRangeTableRelations() in favor of > iterating over a new List field of EState named es_opened_relations, > which is populated by ExecGetRangeTableRelation() with only the > relations that were opened. This speeds up > ExecCloseRangeTableRelations() significantly for the cases with many > runtime-prunable partitions. Here's another version with some cosmetic changes, like fixing some factually incorrect / obsolete comments and typos that I found. I also noticed that I had missed noting near some table_open() that locks taken with those can't possibly invalidate a plan (such as lazily opened partition routing target partitions) and thus need the treatment that locking during execution initialization requires. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v36-0003-Track-opened-range-table-relations-in-a-List-in-.patch (2.3K, 2-v36-0003-Track-opened-range-table-relations-in-a-List-in-.patch) download | inline diff: From d9e140e37ab0dab860ace7c662fba4c0061c7679 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Mon, 13 Mar 2023 15:59:38 +0900 Subject: [PATCH v36 3/3] Track opened range table relations in a List in EState This makes ExecCloseRangeTableRelations faster when there are many relations in the range table but only a few are opened during execution, such as when run-time pruning kicks in on an Append containing 1000s of partition subplans. --- src/backend/executor/execMain.c | 9 +++++---- src/backend/executor/execUtils.c | 2 ++ src/include/nodes/execnodes.h | 2 ++ 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index fd6702a686..506e087474 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -1630,12 +1630,13 @@ ExecCloseResultRelations(EState *estate) void ExecCloseRangeTableRelations(EState *estate) { - int i; + ListCell *lc; - for (i = 0; i < estate->es_range_table_size; i++) + foreach(lc, estate->es_opened_relations) { - if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + Relation rel = lfirst(lc); + + table_close(rel, NoLock); } } diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index a485e7dfc5..f7053072d9 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -829,6 +829,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) } estate->es_relations[rti - 1] = rel; + estate->es_opened_relations = lappend(estate->es_opened_relations, + rel); } return rel; diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index dfa72848c7..984fd2e423 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState Index es_range_table_size; /* size of the range table arrays */ Relation *es_relations; /* Array of per-range-table-entry Relation * pointers, or NULL if not yet opened */ + List *es_opened_relations; /* List of non-NULL entries in + * es_relations in no specific order */ struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ -- 2.35.3 [application/octet-stream] v36-0001-Add-field-to-store-partitioned-relids-to-Append-.patch (20.2K, 3-v36-0001-Add-field-to-store-partitioned-relids-to-Append-.patch) download | inline diff: From 281b73dcf3ee569f70ee112c46cd3bc77258f4aa Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Thu, 9 Mar 2023 11:26:06 +0900 Subject: [PATCH v36 1/3] Add field to store partitioned relids to Append/MergeAppend A future commit would like to move the timing of locking relations referenced in a cached plan to ExecInitNode() traversal of the plan tree from the current loop-over-rangetable in AcquireExecutorLocks(). Given that partitioned tables (their RT indexes) would not be accessible via the new way of finding the relations to lock, add a field to Append/MergeAppend to track them separately. This refactors the code to look up partitioned parent relids from a given list of leaf partition subpaths of an Append/MergeAppend out of make_partition_pruneinfo() into its own function called add_append_subpath_partrelids(). Though, the code needs to be generalized to the cases where child rels can be joinrels or upper (grouping) rels. Also, to make it easier to traverse the parent chain of a child grouping rel, this makes its RelOptInfo.parent to be set, which is already done for baserels and joinrels. --- src/backend/optimizer/plan/createplan.c | 36 +++++-- src/backend/optimizer/plan/planner.c | 3 + src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++ src/backend/partitioning/partprune.c | 124 +++------------------- src/include/nodes/plannodes.h | 14 +++ src/include/optimizer/appendinfo.h | 3 + src/include/partitioning/partprune.h | 3 +- 7 files changed, 194 insertions(+), 123 deletions(-) diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index 910ffbf1e1..794cdb5e3b 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -25,6 +25,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/appendinfo.h" #include "optimizer/clauses.h" #include "optimizer/cost.h" #include "optimizer/optimizer.h" @@ -1209,6 +1210,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) Oid *nodeCollations = NULL; bool *nodeNullsFirst = NULL; bool consider_async = false; + List *allpartrelids = NIL; /* * The subpaths list could be empty, if every child was proven empty by @@ -1350,18 +1352,24 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) ++nasyncplans; } + /* Populate partitioned parent relids. */ + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + plan->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ plan->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1381,7 +1389,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) if (prunequal != NIL) plan->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } plan->appendplans = subplans; @@ -1425,6 +1434,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, List *subplans = NIL; ListCell *subpaths; RelOptInfo *rel = best_path->path.parent; + List *allpartrelids = NIL; /* * We don't have the actual creation of the MergeAppend node split out @@ -1514,18 +1524,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, subplan = (Plan *) sort; } + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + node->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ node->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1537,7 +1552,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, if (prunequal != NIL) node->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } node->mergeplans = subplans; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index a1873ce26d..62b3ec96cc 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -7801,8 +7801,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root, agg_costs, gd, &child_extra, &child_partially_grouped_rel); + /* Mark as child of grouped_rel. */ + child_grouped_rel->parent = grouped_rel; if (child_partially_grouped_rel) { + child_partially_grouped_rel->parent = grouped_rel; partially_grouped_live_children = lappend(partially_grouped_live_children, child_partially_grouped_rel); diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c index 9d377385f1..4876742ab2 100644 --- a/src/backend/optimizer/util/appendinfo.c +++ b/src/backend/optimizer/util/appendinfo.c @@ -40,6 +40,7 @@ static void make_inh_translation_list(Relation oldrelation, AppendRelInfo *appinfo); static Node *adjust_appendrel_attrs_mutator(Node *node, adjust_appendrel_attrs_context *context); +static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); /* @@ -1031,3 +1032,136 @@ distribute_row_identity_vars(PlannerInfo *root) } } } + +/* + * add_append_subpath_partrelids + * Look up a child subpath's rel's partitioned parent relids up to + * parentrel and add the bitmapset containing those into + * 'allpartrelids' + */ +List * +add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids) +{ + RelOptInfo *prel = subpath->parent; + Relids partrelids = NULL; + + /* Nothing to do if there's no parent to begin with. */ + if (!IS_OTHER_REL(prel)) + return allpartrelids; + + /* + * Traverse up to the pathrel's topmost partitioned parent, collecting + * parent relids as we go; but stop if we reach parentrel. (Normally, a + * pathrel's topmost partitioned parent is either parentrel or a UNION ALL + * appendrel child of parentrel. But when handling partitionwise joins of + * multi-level partitioning trees, we can see an append path whose + * parentrel is an intermediate partitioned table.) + */ + do + { + Relids parent_relids = NULL; + + /* + * For simple child rels, we can simply get the parent relid from + * prel->parent. But for partitionwise join and aggregate child rels, + * while we can use prel->parent to move up the tree, parent relids to + * add into 'partrelids' must be found the hard way through the + * AppendInfoInfos, because 1) a joinrel's relids may point to RTE_JOIN + * entries, 2) topmost parent grouping rel's relids field is left NULL. + */ + if (IS_SIMPLE_REL(prel)) + { + prel = prel->parent; + /* Stop once we reach the root partitioned rel. */ + if (!IS_PARTITIONED_REL(prel)) + break; + parent_relids = bms_add_members(parent_relids, prel->relids); + } + else + { + AppendRelInfo **appinfos; + int nappinfos, + i; + + appinfos = find_appinfos_by_relids(root, prel->relids, + &nappinfos); + for (i = 0; i < nappinfos; i++) + { + AppendRelInfo *appinfo = appinfos[i]; + + parent_relids = bms_add_member(parent_relids, + appinfo->parent_relid); + } + pfree(appinfos); + prel = prel->parent; + } + /* accept this level as an interesting parent */ + partrelids = bms_add_members(partrelids, parent_relids); + if (prel == parentrel) + break; /* don't traverse above parentrel */ + } while (IS_OTHER_REL(prel)); + + if (partrelids == NULL) + return allpartrelids; + + return add_part_relids(allpartrelids, partrelids); +} + +/* + * add_part_relids + * Add new info to a list of Bitmapsets of partitioned relids. + * + * Within 'allpartrelids', there is one Bitmapset for each topmost parent + * partitioned rel. Each Bitmapset contains the RT indexes of the topmost + * parent as well as its relevant non-leaf child partitions. Since (by + * construction of the rangetable list) parent partitions must have lower + * RT indexes than their children, we can distinguish the topmost parent + * as being the lowest set bit in the Bitmapset. + * + * 'partrelids' contains the RT indexes of a parent partitioned rel, and + * possibly some non-leaf children, that are newly identified as parents of + * some subpath rel passed to make_partition_pruneinfo(). These are added + * to an appropriate member of 'allpartrelids'. + * + * Note that the list contains only RT indexes of partitioned tables that + * are parents of some scan-level relation appearing in the 'subpaths' that + * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are + * not allowed to be higher than the 'parentrel' associated with the append + * path. In this way, we avoid expending cycles on partitioned rels that + * can't contribute useful pruning information for the problem at hand. + * (It is possible for 'parentrel' to be a child partitioned table, and it + * is also possible for scan-level relations to be child partitioned tables + * rather than leaf partitions. Hence we must construct this relation set + * with reference to the particular append path we're dealing with, rather + * than looking at the full partitioning structure represented in the + * RelOptInfos.) + */ +static List * +add_part_relids(List *allpartrelids, Bitmapset *partrelids) +{ + Index targetpart; + ListCell *lc; + + /* We can easily get the lowest set bit this way: */ + targetpart = bms_next_member(partrelids, -1); + Assert(targetpart > 0); + + /* Look for a matching topmost parent */ + foreach(lc, allpartrelids) + { + Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); + Index currtarget = bms_next_member(currpartrelids, -1); + + if (targetpart == currtarget) + { + /* Found a match, so add any new RT indexes to this hierarchy */ + currpartrelids = bms_add_members(currpartrelids, partrelids); + lfirst(lc) = currpartrelids; + return allpartrelids; + } + } + /* No match, so add the new partition hierarchy to the list */ + return lappend(allpartrelids, partrelids); +} diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index 510145e3c0..3557e07082 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@ -138,7 +138,6 @@ typedef struct PruneStepResult } PruneStepResult; -static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); static List *make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *prunequal, @@ -221,33 +220,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context, * of scan paths for its child rels. * 'prunequal' is a list of potential pruning quals (i.e., restriction * clauses that are applicable to the appendrel). + * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents + * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every + * partition tree involved. */ int make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *subpaths, - List *prunequal) + List *prunequal, + List *allpartrelids) { PartitionPruneInfo *pruneinfo; Bitmapset *allmatchedsubplans = NULL; - List *allpartrelids; List *prunerelinfos; int *relid_subplan_map; ListCell *lc; int i; + Assert(list_length(allpartrelids) > 0); + /* - * Scan the subpaths to see which ones are scans of partition child - * relations, and identify their parent partitioned rels. (Note: we must - * restrict the parent partitioned rels to be parentrel or children of - * parentrel, otherwise we couldn't translate prunequal to match.) - * - * Also construct a temporary array to map from partition-child-relation - * relid to the index in 'subpaths' of the scan plan for that partition. + * Construct a temporary array to map from partition-child-relation relid + * to the index in 'subpaths' of the scan plan for that partition. * (Use of "subplan" rather than "subpath" is a bit of a misnomer, but * we'll let it stand.) For convenience, we use 1-based indexes here, so * that zero can represent an un-filled array entry. */ - allpartrelids = NIL; relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size); i = 1; @@ -256,50 +254,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, Path *path = (Path *) lfirst(lc); RelOptInfo *pathrel = path->parent; - /* We don't consider partitioned joins here */ - if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL) - { - RelOptInfo *prel = pathrel; - Bitmapset *partrelids = NULL; - - /* - * Traverse up to the pathrel's topmost partitioned parent, - * collecting parent relids as we go; but stop if we reach - * parentrel. (Normally, a pathrel's topmost partitioned parent - * is either parentrel or a UNION ALL appendrel child of - * parentrel. But when handling partitionwise joins of - * multi-level partitioning trees, we can see an append path whose - * parentrel is an intermediate partitioned table.) - */ - do - { - AppendRelInfo *appinfo; - - Assert(prel->relid < root->simple_rel_array_size); - appinfo = root->append_rel_array[prel->relid]; - prel = find_base_rel(root, appinfo->parent_relid); - if (!IS_PARTITIONED_REL(prel)) - break; /* reached a non-partitioned parent */ - /* accept this level as an interesting parent */ - partrelids = bms_add_member(partrelids, prel->relid); - if (prel == parentrel) - break; /* don't traverse above parentrel */ - } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL); - - if (partrelids) - { - /* - * Found some relevant parent partitions, which may or may not - * overlap with partition trees we already found. Add new - * information to the allpartrelids list. - */ - allpartrelids = add_part_relids(allpartrelids, partrelids); - /* Also record the subplan in relid_subplan_map[] */ - /* No duplicates please */ - Assert(relid_subplan_map[pathrel->relid] == 0); - relid_subplan_map[pathrel->relid] = i; - } - } + /* No duplicates please */ + Assert(relid_subplan_map[pathrel->relid] == 0); + relid_subplan_map[pathrel->relid] = i; i++; } @@ -368,63 +325,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, return list_length(root->partPruneInfos) - 1; } -/* - * add_part_relids - * Add new info to a list of Bitmapsets of partitioned relids. - * - * Within 'allpartrelids', there is one Bitmapset for each topmost parent - * partitioned rel. Each Bitmapset contains the RT indexes of the topmost - * parent as well as its relevant non-leaf child partitions. Since (by - * construction of the rangetable list) parent partitions must have lower - * RT indexes than their children, we can distinguish the topmost parent - * as being the lowest set bit in the Bitmapset. - * - * 'partrelids' contains the RT indexes of a parent partitioned rel, and - * possibly some non-leaf children, that are newly identified as parents of - * some subpath rel passed to make_partition_pruneinfo(). These are added - * to an appropriate member of 'allpartrelids'. - * - * Note that the list contains only RT indexes of partitioned tables that - * are parents of some scan-level relation appearing in the 'subpaths' that - * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are - * not allowed to be higher than the 'parentrel' associated with the append - * path. In this way, we avoid expending cycles on partitioned rels that - * can't contribute useful pruning information for the problem at hand. - * (It is possible for 'parentrel' to be a child partitioned table, and it - * is also possible for scan-level relations to be child partitioned tables - * rather than leaf partitions. Hence we must construct this relation set - * with reference to the particular append path we're dealing with, rather - * than looking at the full partitioning structure represented in the - * RelOptInfos.) - */ -static List * -add_part_relids(List *allpartrelids, Bitmapset *partrelids) -{ - Index targetpart; - ListCell *lc; - - /* We can easily get the lowest set bit this way: */ - targetpart = bms_next_member(partrelids, -1); - Assert(targetpart > 0); - - /* Look for a matching topmost parent */ - foreach(lc, allpartrelids) - { - Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); - Index currtarget = bms_next_member(currpartrelids, -1); - - if (targetpart == currtarget) - { - /* Found a match, so add any new RT indexes to this hierarchy */ - currpartrelids = bms_add_members(currpartrelids, partrelids); - lfirst(lc) = currpartrelids; - return allpartrelids; - } - } - /* No match, so add the new partition hierarchy to the list */ - return lappend(allpartrelids, partrelids); -} - /* * make_partitionedrel_pruneinfo * Build a List of PartitionedRelPruneInfos, one for each interesting diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 659bd05c0c..a0bb16cff4 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -270,6 +270,13 @@ typedef struct Append List *appendplans; int nasyncplans; /* # of asynchronous plans */ + /* + * RTIs of all partitioned tables whose children are scanned by + * appendplans. The list contains a bitmapset for every partition tree + * covered by this Append. + */ + List *allpartrelids; + /* * All 'appendplans' preceding this index are non-partial plans. All * 'appendplans' from this index onwards are partial plans. @@ -294,6 +301,13 @@ typedef struct MergeAppend List *mergeplans; + /* + * RTIs of all partitioned tables whose children are scanned by + * mergeplans. The list contains a bitmapset for every partition tree + * covered by this MergeAppend. + */ + List *allpartrelids; + /* these fields are just like the sort-key info in struct Sort: */ /* number of sort-key columns */ diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h index a05f91f77d..1621a7319a 100644 --- a/src/include/optimizer/appendinfo.h +++ b/src/include/optimizer/appendinfo.h @@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex, RangeTblEntry *target_rte, Relation target_relation); extern void distribute_row_identity_vars(PlannerInfo *root); +extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids); #endif /* APPENDINFO_H */ diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h index c0d6889d47..2d907d31d4 100644 --- a/src/include/partitioning/partprune.h +++ b/src/include/partitioning/partprune.h @@ -73,7 +73,8 @@ typedef struct PartitionPruneContext extern int make_partition_pruneinfo(struct PlannerInfo *root, struct RelOptInfo *parentrel, List *subpaths, - List *prunequal); + List *prunequal, + List *allpartrelids); extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel); extern Bitmapset *get_matching_partitions(PartitionPruneContext *context, List *pruning_steps); -- 2.35.3 [application/octet-stream] v36-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch (131.3K, 4-v36-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 5defff624f9072b62ec48a7aa0c13faf233bf336 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v36 2/3] Move AcquireExecutorLocks()'s responsibility into the executor This commit introduces a new executor flag EXEC_FLAG_GET_LOCKS that should be passed in eflags to ExecutorStart() if the PlannedStmt comes from a CachedPlan. When set, the executor will take locks on any relations referenced in the plan nodes that need to be initialized for execution. That excludes any partitions that can be pruned during the executor initialization phase, that is, based on the values of only the external (PARAM_EXTERN) parameters. Relations that are not explicitly mentioned in the plan tree, such as views and non-leaf partition parents whose children are mentioned in Append/MergeAppend nodes, are locked separately. After taking each lock, the executor calls CachedPlanStillValid() to check if CachedPlan.is_valid has been reset by PlanCacheRelCallback() due to concurrent modification of relations referenced in the plan. If it is found that the CachedPlan is indeed invalid, the recursive ExecInitNode() traversal is aborted at that point. To allow the proper cleanup of such a partially initialized planstate tree, ExecEndNode() subroutines of various plan nodes have been fixed to account for potentially uninitialized fields. It is the caller's (of ExecutorStart()) responsibility to call ExecutorEnd() even on a QueryDesc containing such a partially initialized PlanState tree. Call sites that use plancache (GetCachedPlan) to get the plan trees to pass to the executor for execution should now be prepared to handle the case that the plan tree may be flagged by the executor as stale as described above. To that end, this commit refactors the relevant code sites to move the ExecutorStart() call closer to the GetCachedPlan() call to reduce the friction in the cases where replanning is needed due to a CachedPlan being marked stale in this manner. Callers must check that QueryDesc.plan_valid is true before passing it on to ExecutorRun() for execution. PortalStart() now performs CreateQueryDesc() and ExecutorStart() for all portal strategies, including those pertaining to multiple queries. The QueryDescs for strategies handled by PortalRunMulti() are remembered in the Portal in a new List field 'qdescs', allocated in a new memory context 'queryContext'. This new arrangment is to make it easier to discard and recreate a Portal if the CachedPlan goes stale during setup. --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 146 +++++--- src/backend/commands/extension.c | 2 + src/backend/commands/matview.c | 3 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 32 +- src/backend/executor/execMain.c | 89 ++++- src/backend/executor/execParallel.c | 8 +- src/backend/executor/execPartition.c | 15 + src/backend/executor/execProcnode.c | 9 + src/backend/executor/execUtils.c | 60 +++- src/backend/executor/functions.c | 2 + src/backend/executor/nodeAgg.c | 23 +- src/backend/executor/nodeAppend.c | 23 +- src/backend/executor/nodeBitmapAnd.c | 10 +- src/backend/executor/nodeBitmapHeapscan.c | 10 +- src/backend/executor/nodeBitmapIndexscan.c | 2 + src/backend/executor/nodeBitmapOr.c | 10 +- src/backend/executor/nodeCtescan.c | 6 +- src/backend/executor/nodeCustom.c | 12 +- src/backend/executor/nodeForeignscan.c | 22 +- src/backend/executor/nodeFunctionscan.c | 3 +- src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 5 +- src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 13 +- src/backend/executor/nodeIncrementalSort.c | 14 +- src/backend/executor/nodeIndexonlyscan.c | 7 +- src/backend/executor/nodeIndexscan.c | 7 +- src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 5 +- src/backend/executor/nodeMemoize.c | 12 +- src/backend/executor/nodeMergeAppend.c | 23 +- src/backend/executor/nodeMergejoin.c | 10 +- src/backend/executor/nodeModifyTable.c | 13 +- .../executor/nodeNamedtuplestorescan.c | 3 +- src/backend/executor/nodeNestloop.c | 7 +- src/backend/executor/nodeProjectSet.c | 5 +- src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 5 +- src/backend/executor/nodeSamplescan.c | 5 +- src/backend/executor/nodeSeqscan.c | 5 +- src/backend/executor/nodeSetOp.c | 5 +- src/backend/executor/nodeSort.c | 8 +- src/backend/executor/nodeSubqueryscan.c | 5 +- src/backend/executor/nodeTableFuncscan.c | 3 +- src/backend/executor/nodeTidrangescan.c | 5 +- src/backend/executor/nodeTidscan.c | 5 +- src/backend/executor/nodeUnique.c | 5 +- src/backend/executor/nodeValuesscan.c | 3 +- src/backend/executor/nodeWindowAgg.c | 55 +++- src/backend/executor/nodeWorktablescan.c | 3 +- src/backend/executor/spi.c | 53 ++- src/backend/nodes/outfuncs.c | 1 + src/backend/nodes/readfuncs.c | 1 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 13 +- src/backend/tcop/pquery.c | 311 ++++++++++-------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 ++------ src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 12 + src/include/nodes/execnodes.h | 2 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 14 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 117 +++++++ .../specs/cached-plan-replan.spec | 50 +++ 82 files changed, 1224 insertions(+), 422 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index beea1ac687..e9f77d5711 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index e57bda7b62..acae5b455b 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -384,6 +384,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -406,12 +407,93 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -515,29 +597,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - Assert(plannedstmt->commandType != CMD_UTILITY); - - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -546,38 +615,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4851,6 +4888,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index 0eabe18335..5a76343123 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -797,11 +797,13 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); ExecutorStart(qdesc, 0); + Assert(qdesc->plan_valid); ExecutorRun(qdesc, ForwardScanDirection, 0, true); ExecutorFinish(qdesc); ExecutorEnd(qdesc); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index fb30d2595c..9adaf6c527 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); /* call ExecutorStart to prepare the plan for execution */ ExecutorStart(queryDesc, 0); + Assert(queryDesc->plan_valid); /* run the plan */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..c9070ed97f 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, it + * must be recreated if portal->plan_valid is false which tells that the + * cached plan was found to have been invalidated when initializing one of + * the plan trees contained in it. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index b32f419176..fd6702a686 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,32 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan, in which case only + * some of the relations referenced in the plan would have been locked; to + * wit, those that AcquirePlannerLocks() deems necessary. Locks necessary + * to fully validate such a plan tree, including relations that are added by + * the planner, will be taken when initializing the plan tree in InitPlan(); + * the the caller must have set the EXEC_FLAG_GET_LOCKS bit in eflags. If the + * CachedPlan gets invalidated as these locks are taken, plan tree + * initialization is suspended at the point when such invalidation is first + * detected and InitPlan() returns after setting queryDesc->plan_valid to + * false. queryDesc->planstate would be pointing to a potentially partially + * initialized PlanState tree in that case. Callers must retry the execution + * with a freshly created CachedPlan in that case, after properly freeing the + * partially valid QueryDesc. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + (CachedPlanStillValid(queryDesc->cplan) && + (eflags & EXEC_FLAG_GET_LOCKS) != 0)); + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +603,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +816,19 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } - /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -801,20 +839,32 @@ InitPlan(QueryDesc *queryDesc, int eflags) Plan *plan = plannedstmt->planTree; List *rangeTable = plannedstmt->rtable; EState *estate = queryDesc->estate; - PlanState *planstate; + PlanState *planstate = NULL; TupleDesc tupType; ListCell *l; int i; /* - * Do permissions checks + * Set up range table in EState. */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + + /* Make sure ExecPlanStillValid() can work. */ + estate->es_cachedplan = queryDesc->cplan; /* - * initialize the node's execution state + * Lock any views that were mentioned in the query if needed. View + * relations must be locked separately like this, because they are not + * referenced in the plan tree. */ - ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + ExecLockViewRelations(plannedstmt->viewRelations, estate); + if (!ExecPlanStillValid(estate)) + goto failed; + + /* + * Do permissions checks + */ + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); estate->es_plannedstmt = plannedstmt; estate->es_part_prune_infos = plannedstmt->partPruneInfos; @@ -849,6 +899,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -919,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); + if (!ExecPlanStillValid(estate)) + goto failed; i++; } @@ -929,6 +983,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -972,6 +1028,17 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such. Note that we do + * set planstate, even if it may only be partially initialized, so that + * ExecEndPlan() can process it. + */ + queryDesc->planstate = planstate; + queryDesc->plan_valid = false; } /* @@ -1389,7 +1456,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -2797,7 +2864,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2884,6 +2952,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } @@ -2937,6 +3006,10 @@ EvalPlanQualEnd(EPQState *epqstate) MemoryContext oldcontext; ListCell *l; + /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */ + if (epqstate->parentestate == NULL) + return; + rtsize = epqstate->parentestate->es_range_table_size; /* diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..df4cc5ddaf 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); @@ -1432,6 +1437,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc) /* Start up the executor */ queryDesc->plannedstmt->jitFlags = fpes->jit_flags; ExecutorStart(queryDesc, fpes->eflags); + Assert(queryDesc->plan_valid); /* Special executor initialization steps for parallel workers */ queryDesc->planstate->state->es_query_dsa = area; diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index fd6ca8a5d9..b8580e98f7 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -513,6 +513,12 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate, oldcxt = MemoryContextSwitchTo(proute->memcxt); + /* + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. + */ partrel = table_open(partOid, RowExclusiveLock); leaf_part_rri = makeNode(ResultRelInfo); @@ -1111,6 +1117,11 @@ ExecInitPartitionDispatchInfo(EState *estate, * Only sub-partitioned tables need to be locked here. The root * partitioned table will already have been locked as it's referenced in * the query's rtable. + * + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. */ if (partoid != RelationGetRelid(proute->partition_root)) rel = table_open(partoid, RowExclusiveLock); @@ -1817,6 +1828,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1943,6 +1956,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..6f3c37b6fd 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return result; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -403,6 +406,12 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); subps = lappend(subps, sstate); + if (!ExecPlanStillValid(estate)) + { + /* Don't lose track of those initialized. */ + result->initPlan = subps; + return result; + } } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 012dbb6965..a485e7dfc5 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -804,7 +804,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -833,6 +834,61 @@ ExecGetRangeTableRelation(EState *estate, Index rti) return rel; } +/* + * ExecLockViewRelations + * Lock view relations, if any, in a given query + */ +void +ExecLockViewRelations(List *viewRelations, EState *estate) +{ + ListCell *lc; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + LockRelationOid(rte->relid, rte->rellockmode); + } +} + +/* + * ExecLockAppendNonLeafRelations + * Lock non-leaf relations whose children are scanned by a given + * Append/MergeAppend node + */ +void +ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids) +{ + ListCell *l; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(l, allpartrelids) + { + Bitmapset *partrelids = lfirst_node(Bitmapset, l); + int i; + + i = -1; + while ((i = bms_next_member(partrelids, i)) > 0) + { + RangeTblEntry *rte = exec_rt_fetch(i, estate); + + LockRelationOid(rte->relid, rte->rellockmode); + } + } +} + /* * ExecInitResultRelation * Open relation given by the passed-in RT index and fill its @@ -848,6 +904,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index 50e06ec693..f8c9de1fda 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -843,6 +843,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, @@ -868,6 +869,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) else eflags = 0; /* default run-to-completion flags */ ExecutorStart(es->qd, eflags); + Assert(es->qd->plan_valid); } es->status = F_EXEC_RUN; diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 19342a420c..06e0d7d149 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3134,15 +3134,18 @@ hashagg_reset_spill_state(AggState *aggstate) { HashAggSpill *spill = &aggstate->hash_spills[setno]; - pfree(spill->ntuples); - pfree(spill->partitions); + if (spill->ntuples) + pfree(spill->ntuples); + if (spill->partitions) + pfree(spill->partitions); } pfree(aggstate->hash_spills); aggstate->hash_spills = NULL; } /* free batches */ - list_free_deep(aggstate->hash_batches); + if (aggstate->hash_batches) + list_free_deep(aggstate->hash_batches); aggstate->hash_batches = NIL; /* close tape set */ @@ -3296,6 +3299,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return aggstate; /* * initialize source tuple type. @@ -4336,10 +4341,13 @@ ExecEndAgg(AggState *node) { AggStatePerTrans pertrans = &node->pertrans[transno]; - for (setno = 0; setno < numGroupingSets; setno++) + if (pertrans) { - if (pertrans->sortstates[setno]) - tuplesort_end(pertrans->sortstates[setno]); + for (setno = 0; setno < numGroupingSets; setno++) + { + if (pertrans->sortstates[setno]) + tuplesort_end(pertrans->sortstates[setno]); + } } } @@ -4357,7 +4365,8 @@ ExecEndAgg(AggState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index c185b11c67..091f979c46 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -109,10 +109,11 @@ AppendState * ExecInitAppend(Append *node, EState *estate, int eflags) { AppendState *appendstate = makeNode(AppendState); - PlanState **appendplanstates; + PlanState **appendplanstates = NULL; Bitmapset *validsubplans; Bitmapset *asyncplans; int nplans; + int ninited = 0; int nasyncplans; int firstvalid; int i, @@ -133,6 +134,15 @@ ExecInitAppend(Append *node, EState *estate, int eflags) appendstate->as_syncdone = false; appendstate->as_begun = false; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -148,6 +158,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + goto early_exit; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -222,11 +234,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } appendstate->as_first_partial_plan = firstvalid; - appendstate->appendplans = appendplanstates; - appendstate->as_nplans = nplans; /* Initialize async state */ appendstate->as_asyncplans = asyncplans; @@ -276,6 +289,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags) /* For parallel query, this will be overridden later. */ appendstate->choose_next_subplan = choose_next_subplan_locally; +early_exit: + appendstate->appendplans = appendplanstates; + appendstate->as_nplans = ninited; + return appendstate; } diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..acc6c50e20 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -57,6 +57,7 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) BitmapAndState *bitmapandstate = makeNode(BitmapAndState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -77,8 +78,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) bitmapandstate->ps.plan = (Plan *) node; bitmapandstate->ps.state = estate; bitmapandstate->ps.ExecProcNode = ExecBitmapAnd; - bitmapandstate->bitmapplans = bitmapplanstates; - bitmapandstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -89,6 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -99,6 +101,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmapandstate->bitmapplans = bitmapplanstates; + bitmapandstate->nplans = ninited; + return bitmapandstate; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..e6a689eefb 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -665,7 +665,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subplans @@ -693,7 +694,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) /* * close heap scan */ - table_endscan(scanDesc); + if (scanDesc) + table_endscan(scanDesc); } /* ---------------------------------------------------------------- @@ -763,11 +765,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c index 83ec9ede89..cc8332ef68 100644 --- a/src/backend/executor/nodeBitmapIndexscan.c +++ b/src/backend/executor/nodeBitmapIndexscan.c @@ -263,6 +263,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->biss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..babad1b4b2 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -58,6 +58,7 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) BitmapOrState *bitmaporstate = makeNode(BitmapOrState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -78,8 +79,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) bitmaporstate->ps.plan = (Plan *) node; bitmaporstate->ps.state = estate; bitmaporstate->ps.ExecProcNode = ExecBitmapOr; - bitmaporstate->bitmapplans = bitmapplanstates; - bitmaporstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -90,6 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -100,6 +102,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmaporstate->bitmapplans = bitmapplanstates; + bitmaporstate->nplans = ninited; + return bitmaporstate; } diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c index cc4c4243e2..eed5b75a4f 100644 --- a/src/backend/executor/nodeCtescan.c +++ b/src/backend/executor/nodeCtescan.c @@ -297,14 +297,16 @@ ExecEndCteScan(CteScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * If I am the leader, free the tuplestore. */ if (node->leader == node) { - tuplestore_end(node->cte_table); + if (node->cte_table) + tuplestore_end(node->cte_table); node->cte_table = NULL; } } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..b03499fae5 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return css; css->ss.ss_currentRelation = scan_rel; } @@ -127,6 +129,10 @@ ExecCustomScan(PlanState *pstate) void ExecEndCustomScan(CustomScanState *node) { + /* + * XXX - BeginCustomScan() may not have occurred if ExecInitCustomScan() + * hit the early exit case. + */ Assert(node->methods->EndCustomScan != NULL); node->methods->EndCustomScan(node); @@ -134,8 +140,10 @@ ExecEndCustomScan(CustomScanState *node) ExecFreeExprContext(&node->ss.ps); /* Clean out the tuple table */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } void diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..d3f0a65485 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * Tell the FDW to initialize the scan. @@ -300,14 +304,17 @@ ExecEndForeignScan(ForeignScanState *node) ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; EState *estate = node->ss.ps.state; - /* Let the FDW shut down */ - if (plan->operation != CMD_SELECT) + /* Let the FDW shut down if needed. */ + if (node->fdw_state) { - if (estate->es_epq_active == NULL) - node->fdwroutine->EndDirectModify(node); + if (plan->operation != CMD_SELECT) + { + if (estate->es_epq_active == NULL) + node->fdwroutine->EndDirectModify(node); + } + else + node->fdwroutine->EndForeignScan(node); } - else - node->fdwroutine->EndForeignScan(node); /* Shut down any outer plan. */ if (outerPlanState(node)) @@ -319,7 +326,8 @@ ExecEndForeignScan(ForeignScanState *node) /* clean out the tuple table */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c index dd06ef8aee..792ecda4a9 100644 --- a/src/backend/executor/nodeFunctionscan.c +++ b/src/backend/executor/nodeFunctionscan.c @@ -533,7 +533,8 @@ ExecEndFunctionScan(FunctionScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release slots and tuplestore resources diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..365d3af3e4 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gatherstate; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..8d2809f079 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gm_state; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..e0832bb778 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return grpstate; /* * Initialize scan slot and type. @@ -231,7 +233,8 @@ ExecEndGroup(GroupState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index 1e624fed7a..a8966f8b4a 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return hashstate; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index 32f12fefd7..1448a7ddba 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -689,8 +689,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* @@ -811,9 +815,12 @@ ExecEndHashJoin(HashJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->hj_OuterTupleSlot); - ExecClearTuple(node->hj_HashTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->hj_OuterTupleSlot) + ExecClearTuple(node->hj_OuterTupleSlot); + if (node->hj_HashTupleSlot) + ExecClearTuple(node->hj_HashTupleSlot); /* * clean up subtrees diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..6b2da56044 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return incrsortstate; /* * Initialize scan slot and type. @@ -1080,12 +1082,16 @@ ExecEndIncrementalSort(IncrementalSortState *node) SO_printf("ExecEndIncrementalSort: shutting down sort node\n"); /* clean out the scan tuple */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* must drop standalone tuple slots from outer node */ - ExecDropSingleTupleTableSlot(node->group_pivot); - ExecDropSingleTupleTableSlot(node->transfer_tuple); + if (node->group_pivot) + ExecDropSingleTupleTableSlot(node->group_pivot); + if (node->transfer_tuple) + ExecDropSingleTupleTableSlot(node->transfer_tuple); /* * Release tuplesort resources. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..b60a086464 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -394,7 +394,8 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if(node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -565,6 +568,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..628c233919 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -808,7 +808,8 @@ ExecEndIndexScan(IndexScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -970,6 +973,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->iss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..2fcbde74ed 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return limitstate; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..3a8aa2b5a4 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return lrstate; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..f146ebb1d7 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return matstate; /* * Initialize result type and slot. No need to initialize projection info @@ -242,7 +244,8 @@ ExecEndMaterial(MaterialState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 4f04269e26..3003ee1e5c 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mstate; /* * Initialize return slot and type. No need to initialize projection info @@ -1043,6 +1045,7 @@ ExecEndMemoize(MemoizeState *node) { #ifdef USE_ASSERT_CHECKING /* Validate the memory accounting code is correct in assert builds. */ + if (node->hashtable) { int count; uint64 mem = 0; @@ -1089,11 +1092,14 @@ ExecEndMemoize(MemoizeState *node) } /* Remove the cache context */ - MemoryContextDelete(node->tableContext); + if (node->tableContext) + MemoryContextDelete(node->tableContext); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to cache result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * free exprcontext diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..40bba35499 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -65,9 +65,10 @@ MergeAppendState * ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) { MergeAppendState *mergestate = makeNode(MergeAppendState); - PlanState **mergeplanstates; + PlanState **mergeplanstates = NULL; Bitmapset *validsubplans; int nplans; + int ninited = 0; int i, j; @@ -81,6 +82,15 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) mergestate->ps.state = estate; mergestate->ps.ExecProcNode = ExecMergeAppend; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -96,6 +106,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return mergestate; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -122,8 +134,6 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) } mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *)); - mergestate->mergeplans = mergeplanstates; - mergestate->ms_nplans = nplans; mergestate->ms_slots = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans); mergestate->ms_heap = binaryheap_allocate(nplans, heap_compare_slots, @@ -152,6 +162,9 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } mergestate->ps.ps_ProjInfo = NULL; @@ -188,6 +201,10 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) */ mergestate->ms_initialized = false; +early_exit: + mergestate->mergeplans = mergeplanstates; + mergestate->ms_nplans = ninited; + return mergestate; } diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..968be05568 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return mergestate; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* @@ -1642,8 +1646,10 @@ ExecEndMergeJoin(MergeJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->mj_MarkedTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->mj_MarkedTupleSlot) + ExecClearTuple(node->mj_MarkedTupleSlot); /* * shut down the subplans diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index 3a67389508..b07d7cac28 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -3922,6 +3922,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) Plan *subplan = outerPlan(node); CmdType operation = node->operation; int nrels = list_length(node->resultRelations); + int ninited = 0; ResultRelInfo *resultRelInfo; List *arowmarks; ListCell *l; @@ -3943,7 +3944,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) mtstate->canSetTag = node->canSetTag; mtstate->mt_done = false; - mtstate->mt_nrels = nrels; mtstate->resultRelInfo = (ResultRelInfo *) palloc(nrels * sizeof(ResultRelInfo)); @@ -3978,6 +3978,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -4004,6 +4007,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * For child result relations, store the root result relation @@ -4031,11 +4036,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * Do additional per-result-relation initialization. */ - for (i = 0; i < nrels; i++) + for (i = 0; i < nrels; i++, ninited++) { resultRelInfo = &mtstate->resultRelInfo[i]; @@ -4384,6 +4391,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) estate->es_auxmodifytables = lcons(mtstate, estate->es_auxmodifytables); +early_exit: + mtstate->mt_nrels = ninited; return mtstate; } diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c index 46832ad82f..1f92c43d3b 100644 --- a/src/backend/executor/nodeNamedtuplestorescan.c +++ b/src/backend/executor/nodeNamedtuplestorescan.c @@ -174,7 +174,8 @@ ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..deda0c2559 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; /* * Initialize result slot, type and projection. @@ -372,7 +376,8 @@ ExecEndNestLoop(NestLoopState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); /* * close down subplans diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..85d20c4680 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return state; /* * we don't use inner plan @@ -328,7 +330,8 @@ ExecEndProjectSet(ProjectSetState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..967fe4f287 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..c549b684a3 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return resstate; /* * we don't use inner plan @@ -248,7 +250,8 @@ ExecEndResult(ResultState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..b3bc9b1f77 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; @@ -198,7 +200,8 @@ ExecEndSampleScan(SampleScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..e7ca19ee4e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, @@ -200,7 +202,8 @@ ExecEndSeqScan(SeqScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..95950a5c20 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return setopstate; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* @@ -583,7 +585,8 @@ void ExecEndSetOp(SetOpState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* free subsidiary stuff including hashtable */ if (node->tableContext) diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..89fef86aba 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return sortstate; /* * Initialize scan slot and type. @@ -306,9 +308,11 @@ ExecEndSort(SortState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * Release tuplesort resources diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..9b8cddc89f 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return subquerystate; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) @@ -177,7 +179,8 @@ ExecEndSubqueryScan(SubqueryScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subquery diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c index 0c6c912778..d7536953f1 100644 --- a/src/backend/executor/nodeTableFuncscan.c +++ b/src/backend/executor/nodeTableFuncscan.c @@ -223,7 +223,8 @@ ExecEndTableFuncScan(TableFuncScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..1ae451d7a6 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -342,7 +342,8 @@ ExecEndTidRangeScan(TidRangeScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -386,6 +387,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidrangestate; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 862bd0330b..9fe76b1c60 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -483,7 +483,8 @@ ExecEndTidScan(TidScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -529,6 +530,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidstate; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..69f23b02c6 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return uniquestate; /* * Initialize result slot and type. Unique nodes do no projections, so @@ -169,7 +171,8 @@ void ExecEndUnique(UniqueState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); ExecFreeExprContext(&node->ps); diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c index 32ace63017..f5dedbab63 100644 --- a/src/backend/executor/nodeValuesscan.c +++ b/src/backend/executor/nodeValuesscan.c @@ -340,7 +340,8 @@ ExecEndValuesScan(ValuesScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 7c07fb0684..616bb97675 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -1334,7 +1334,7 @@ release_partition(WindowAggState *winstate) WindowStatePerFunc perfuncstate = &(winstate->perfunc[i]); /* Release any partition-local state of this window function */ - if (perfuncstate->winobj) + if (perfuncstate && perfuncstate->winobj) perfuncstate->winobj->localmem = NULL; } @@ -1344,12 +1344,17 @@ release_partition(WindowAggState *winstate) * any aggregate temp data). We don't rely on retail pfree because some * aggregates might have allocated data we don't have direct pointers to. */ - MemoryContextResetAndDeleteChildren(winstate->partcontext); - MemoryContextResetAndDeleteChildren(winstate->aggcontext); - for (i = 0; i < winstate->numaggs; i++) + if (winstate->partcontext) + MemoryContextResetAndDeleteChildren(winstate->partcontext); + if (winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->aggcontext); + if (winstate->peragg) { - if (winstate->peragg[i].aggcontext != winstate->aggcontext) - MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + for (i = 0; i < winstate->numaggs; i++) + { + if (winstate->peragg[i].aggcontext != winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + } } if (winstate->buffer) @@ -2451,6 +2456,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return winstate; /* * initialize source tuple type (which is also the tuple type that we'll @@ -2679,11 +2686,16 @@ ExecEndWindowAgg(WindowAggState *node) release_partition(node); - ExecClearTuple(node->ss.ss_ScanTupleSlot); - ExecClearTuple(node->first_part_slot); - ExecClearTuple(node->agg_row_slot); - ExecClearTuple(node->temp_slot_1); - ExecClearTuple(node->temp_slot_2); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->first_part_slot) + ExecClearTuple(node->first_part_slot); + if (node->agg_row_slot) + ExecClearTuple(node->agg_row_slot); + if (node->temp_slot_1) + ExecClearTuple(node->temp_slot_1); + if (node->temp_slot_2) + ExecClearTuple(node->temp_slot_2); if (node->framehead_slot) ExecClearTuple(node->framehead_slot); if (node->frametail_slot) @@ -2696,16 +2708,23 @@ ExecEndWindowAgg(WindowAggState *node) node->ss.ps.ps_ExprContext = node->tmpcontext; ExecFreeExprContext(&node->ss.ps); - for (i = 0; i < node->numaggs; i++) + if (node->peragg) { - if (node->peragg[i].aggcontext != node->aggcontext) - MemoryContextDelete(node->peragg[i].aggcontext); + for (i = 0; i < node->numaggs; i++) + { + if (node->peragg[i].aggcontext != node->aggcontext) + MemoryContextDelete(node->peragg[i].aggcontext); + } } - MemoryContextDelete(node->partcontext); - MemoryContextDelete(node->aggcontext); + if (node->partcontext) + MemoryContextDelete(node->partcontext); + if (node->aggcontext) + MemoryContextDelete(node->aggcontext); - pfree(node->perfunc); - pfree(node->peragg); + if (node->perfunc) + pfree(node->perfunc); + if (node->peragg) + pfree(node->peragg); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c index 0c13448236..d70c6afde3 100644 --- a/src/backend/executor/nodeWorktablescan.c +++ b/src/backend/executor/nodeWorktablescan.c @@ -200,7 +200,8 @@ ExecEndWorkTableScan(WorkTableScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index e3a170c38b..26a9ea342a 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2668,14 +2680,36 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + + /* Take locks if using a CachedPlan */ + if (qdesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + Assert(cplan); + ReleaseCachedPlan(cplan, plan_owner); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2850,10 +2884,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2897,14 +2930,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index ba00b99249..955286513d 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -513,6 +513,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ WRITE_OID_FIELD(relid); + WRITE_CHAR_FIELD(relkind); WRITE_INT_FIELD(rellockmode); WRITE_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index 597e5b3ea8..a136ae1d60 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -503,6 +503,7 @@ _readRangeTblEntry(void) READ_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ READ_OID_FIELD(relid); + READ_CHAR_FIELD(relkind); READ_INT_FIELD(rellockmode); READ_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 62b3ec96cc..5f3ffd98af 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 5cc8366af6..f13240bf33 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -604,6 +605,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index 980dc1816f..1631c8b993 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1849,11 +1849,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..6d0ea07801 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1199,6 +1199,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1703,6 +1704,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1994,10 +1996,19 @@ exec_bind_message(StringInfo input_message) PopActiveSnapshot(); /* - * And we're ready to start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..c93a950d7f 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt, qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +353,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +419,56 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti(). */ + if (portal->strategy == PORTAL_ONE_RETURNING || + portal->strategy == PORTAL_ONE_MOD_WITH) + portal->qdescs = list_make1(queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +476,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +500,90 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool first = true; + + /* Take locks if using a CachedPlan */ + myeflags = 0; + if (portal->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* + * Push the snapshot to be used by the executor. + */ + if (!is_utility) + { + /* + * Must copy the snapshot if we'll need to update + * its command ID. + */ + if (!first) + PushCopiedSnapshot(GetTransactionSnapshot()); + else + PushActiveSnapshot(GetTransactionSnapshot()); + } + + /* + * From the 2nd statement onwards, update the command + * ID and the snapshot to match. + */ + if (!first) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + first = false; + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + !is_utility ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + if (is_utility) + continue; + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + ExecutorStart(queryDesc, myeflags); + PopActiveSnapshot(); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + portal->plan_valid = false; + goto early_exit; + } + } + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +595,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1193,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1214,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1271,23 +1272,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1362,15 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c07382051d..38ae43e24b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 7c1071ddd1..da39b2e4ff 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -87,7 +87,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -103,6 +107,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index dbd77050c7..ebb6665950 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -59,6 +60,8 @@ #define EXEC_FLAG_MARK 0x0008 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0010 /* skip AfterTrigger calls */ #define EXEC_FLAG_WITH_NO_DATA 0x0020 /* rel scannability doesn't matter */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should the executor lock + * relations? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -245,6 +248,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode @@ -579,6 +589,8 @@ exec_rt_fetch(Index rti, EState *estate) } extern Relation ExecGetRangeTableRelation(EState *estate, Index rti); +extern void ExecLockViewRelations(List *viewRelations, EState *estate); +extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids); extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Index rti); diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index d97f5a8e7d..dfa72848c7 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -623,6 +623,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d61a62da19..9b888b0d75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index a0bb16cff4..7cae624bbd 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..8990fe72e3 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor on every relation lock taken when initializing the + * plan tree in the CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..4f450b9d9b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,117 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +-------------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo11 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo11_a_idx + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +----------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo11 foo_1 + Filter: (a = $1) +(4 rows) + + +starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock +step s1prep2: SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Index Only Scan using foo11_a_idx on foo11 t1 + -> Materialize + -> Index Scan using foo11_a_idx on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(18 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec2: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec2: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +--------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Sort + Sort Key: t1.a + -> Seq Scan on foo11 t1 + -> Sort + Sort Key: t2.a + -> Seq Scan on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(21 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..67cfed7044 --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,50 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a); + CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1); + CREATE INDEX foo11_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +step "s1prep2" { SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +step "s1exec2" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo11_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" +permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock" -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-03-27 08:18 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-03-27 08:18 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> On Wed, Mar 22, 2023 at 9:48 PM Amit Langote <[email protected]> wrote: > On Tue, Mar 14, 2023 at 7:07 PM Amit Langote <[email protected]> wrote: > > On Thu, Mar 2, 2023 at 10:52 PM Amit Langote <[email protected]> wrote: > > > I think I have figured out what might be going wrong on that cfbot > > > animal after building with the same CPPFLAGS as that animal locally. > > > I had forgotten to update _out/_readRangeTblEntry() to account for the > > > patch's change that a view's RTE_SUBQUERY now also preserves relkind > > > in addition to relid and rellockmode for the locking consideration. > > > > > > Also, I noticed that a multi-query Portal execution with rules was > > > failing (thanks to a regression test added in a7d71c41db) because of > > > the snapshot used for the 2nd query onward not being updated for > > > command ID change under patched model of multi-query Portal execution. > > > To wit, under the patched model, all queries in the multi-query Portal > > > case undergo ExecutorStart() before any of it is run with > > > ExecutorRun(). The patch hadn't changed things however to update the > > > snapshot's command ID for the 2nd query onwards, which caused the > > > aforementioned test case to fail. > > > > > > This new model does however mean that the 2nd query onwards must use > > > PushCopiedSnapshot() given the current requirement of > > > UpdateActiveSnapshotCommandId() that the snapshot passed to it must > > > not be referenced anywhere else. The new model basically requires > > > that each query's QueryDesc points to its own copy of the > > > ActiveSnapshot. That may not be a thing in favor of the patched model > > > though. For now, I haven't been able to come up with a better > > > alternative. > > > > Here's a new version addressing the following 2 points. > > > > * Like views, I realized that non-leaf relations of partition trees > > scanned by an Append/MergeAppend would need to be locked separately, > > because ExecInitNode() traversal of the plan tree would not account > > for them. That is, they are not opened using > > ExecGetRangeTableRelation() or ExecOpenScanRelation(). One exception > > is that some (if not all) of those non-leaf relations may be > > referenced in PartitionPruneInfo and so locked as part of initializing > > the corresponding PartitionPruneState, but I decided not to complicate > > the code to filter out such relations from the set locked separately. > > To carry the set of relations to lock, the refactoring patch 0001 > > re-introduces the List of Bitmapset field named allpartrelids into > > Append/MergeAppend nodes, which we had previously removed on the > > grounds that those relations need not be locked separately (commits > > f2343653f5b, f003a7522bf). > > > > * I decided to initialize QueryDesc.planstate even in the cases where > > ExecInitNode() traversal is aborted in the middle on detecting > > CachedPlan invalidation such that it points to a partially initialized > > PlanState tree. My earlier thinking that each PlanState node need not > > be visited for resource cleanup in such cases was naive after all. To > > that end, I've fixed the ExecEndNode() subroutines of all Plan node > > types to account for potentially uninitialized fields. There are a > > couple of cases where I'm a bit doubtful though. In > > ExecEndCustomScan(), there's no indication in CustomScanState whether > > it's OK to call EndCustomScan() when BeginCustomScan() may not have > > been called. For ForeignScanState, I've assumed that > > ForeignScanState.fdw_state being set can be used as a marker that > > BeginForeignScan would have been called, though maybe that's not a > > solid assumption. > > > > I'm also attaching a new (small) patch 0003 that eliminates the > > loop-over-rangetable in ExecCloseRangeTableRelations() in favor of > > iterating over a new List field of EState named es_opened_relations, > > which is populated by ExecGetRangeTableRelation() with only the > > relations that were opened. This speeds up > > ExecCloseRangeTableRelations() significantly for the cases with many > > runtime-prunable partitions. > > Here's another version with some cosmetic changes, like fixing some > factually incorrect / obsolete comments and typos that I found. I > also noticed that I had missed noting near some table_open() that > locks taken with those can't possibly invalidate a plan (such as > lazily opened partition routing target partitions) and thus need the > treatment that locking during execution initialization requires. Rebased over 3c05284d83b2 ("Invent GENERIC_PLAN option for EXPLAIN."). -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v37-0001-Add-field-to-store-partitioned-relids-to-Append-.patch (20.2K, 2-v37-0001-Add-field-to-store-partitioned-relids-to-Append-.patch) download | inline diff: From dfc41510ef3ebec38e7a56b639ffa41193109b43 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Thu, 9 Mar 2023 11:26:06 +0900 Subject: [PATCH v37 1/3] Add field to store partitioned relids to Append/MergeAppend A future commit would like to move the timing of locking relations referenced in a cached plan to ExecInitNode() traversal of the plan tree from the current loop-over-rangetable in AcquireExecutorLocks(). Given that partitioned tables (their RT indexes) would not be accessible via the new way of finding the relations to lock, add a field to Append/MergeAppend to track them separately. This refactors the code to look up partitioned parent relids from a given list of leaf partition subpaths of an Append/MergeAppend out of make_partition_pruneinfo() into its own function called add_append_subpath_partrelids(). Though, the code needs to be generalized to the cases where child rels can be joinrels or upper (grouping) rels. Also, to make it easier to traverse the parent chain of a child grouping rel, this makes its RelOptInfo.parent to be set, which is already done for baserels and joinrels. --- src/backend/optimizer/plan/createplan.c | 36 +++++-- src/backend/optimizer/plan/planner.c | 3 + src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++ src/backend/partitioning/partprune.c | 124 +++------------------- src/include/nodes/plannodes.h | 14 +++ src/include/optimizer/appendinfo.h | 3 + src/include/partitioning/partprune.h | 3 +- 7 files changed, 194 insertions(+), 123 deletions(-) diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index 910ffbf1e1..794cdb5e3b 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -25,6 +25,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/appendinfo.h" #include "optimizer/clauses.h" #include "optimizer/cost.h" #include "optimizer/optimizer.h" @@ -1209,6 +1210,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) Oid *nodeCollations = NULL; bool *nodeNullsFirst = NULL; bool consider_async = false; + List *allpartrelids = NIL; /* * The subpaths list could be empty, if every child was proven empty by @@ -1350,18 +1352,24 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) ++nasyncplans; } + /* Populate partitioned parent relids. */ + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + plan->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ plan->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1381,7 +1389,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) if (prunequal != NIL) plan->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } plan->appendplans = subplans; @@ -1425,6 +1434,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, List *subplans = NIL; ListCell *subpaths; RelOptInfo *rel = best_path->path.parent; + List *allpartrelids = NIL; /* * We don't have the actual creation of the MergeAppend node split out @@ -1514,18 +1524,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, subplan = (Plan *) sort; } + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + node->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ node->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1537,7 +1552,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, if (prunequal != NIL) node->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } node->mergeplans = subplans; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index a1873ce26d..62b3ec96cc 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -7801,8 +7801,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root, agg_costs, gd, &child_extra, &child_partially_grouped_rel); + /* Mark as child of grouped_rel. */ + child_grouped_rel->parent = grouped_rel; if (child_partially_grouped_rel) { + child_partially_grouped_rel->parent = grouped_rel; partially_grouped_live_children = lappend(partially_grouped_live_children, child_partially_grouped_rel); diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c index 9d377385f1..4876742ab2 100644 --- a/src/backend/optimizer/util/appendinfo.c +++ b/src/backend/optimizer/util/appendinfo.c @@ -40,6 +40,7 @@ static void make_inh_translation_list(Relation oldrelation, AppendRelInfo *appinfo); static Node *adjust_appendrel_attrs_mutator(Node *node, adjust_appendrel_attrs_context *context); +static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); /* @@ -1031,3 +1032,136 @@ distribute_row_identity_vars(PlannerInfo *root) } } } + +/* + * add_append_subpath_partrelids + * Look up a child subpath's rel's partitioned parent relids up to + * parentrel and add the bitmapset containing those into + * 'allpartrelids' + */ +List * +add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids) +{ + RelOptInfo *prel = subpath->parent; + Relids partrelids = NULL; + + /* Nothing to do if there's no parent to begin with. */ + if (!IS_OTHER_REL(prel)) + return allpartrelids; + + /* + * Traverse up to the pathrel's topmost partitioned parent, collecting + * parent relids as we go; but stop if we reach parentrel. (Normally, a + * pathrel's topmost partitioned parent is either parentrel or a UNION ALL + * appendrel child of parentrel. But when handling partitionwise joins of + * multi-level partitioning trees, we can see an append path whose + * parentrel is an intermediate partitioned table.) + */ + do + { + Relids parent_relids = NULL; + + /* + * For simple child rels, we can simply get the parent relid from + * prel->parent. But for partitionwise join and aggregate child rels, + * while we can use prel->parent to move up the tree, parent relids to + * add into 'partrelids' must be found the hard way through the + * AppendInfoInfos, because 1) a joinrel's relids may point to RTE_JOIN + * entries, 2) topmost parent grouping rel's relids field is left NULL. + */ + if (IS_SIMPLE_REL(prel)) + { + prel = prel->parent; + /* Stop once we reach the root partitioned rel. */ + if (!IS_PARTITIONED_REL(prel)) + break; + parent_relids = bms_add_members(parent_relids, prel->relids); + } + else + { + AppendRelInfo **appinfos; + int nappinfos, + i; + + appinfos = find_appinfos_by_relids(root, prel->relids, + &nappinfos); + for (i = 0; i < nappinfos; i++) + { + AppendRelInfo *appinfo = appinfos[i]; + + parent_relids = bms_add_member(parent_relids, + appinfo->parent_relid); + } + pfree(appinfos); + prel = prel->parent; + } + /* accept this level as an interesting parent */ + partrelids = bms_add_members(partrelids, parent_relids); + if (prel == parentrel) + break; /* don't traverse above parentrel */ + } while (IS_OTHER_REL(prel)); + + if (partrelids == NULL) + return allpartrelids; + + return add_part_relids(allpartrelids, partrelids); +} + +/* + * add_part_relids + * Add new info to a list of Bitmapsets of partitioned relids. + * + * Within 'allpartrelids', there is one Bitmapset for each topmost parent + * partitioned rel. Each Bitmapset contains the RT indexes of the topmost + * parent as well as its relevant non-leaf child partitions. Since (by + * construction of the rangetable list) parent partitions must have lower + * RT indexes than their children, we can distinguish the topmost parent + * as being the lowest set bit in the Bitmapset. + * + * 'partrelids' contains the RT indexes of a parent partitioned rel, and + * possibly some non-leaf children, that are newly identified as parents of + * some subpath rel passed to make_partition_pruneinfo(). These are added + * to an appropriate member of 'allpartrelids'. + * + * Note that the list contains only RT indexes of partitioned tables that + * are parents of some scan-level relation appearing in the 'subpaths' that + * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are + * not allowed to be higher than the 'parentrel' associated with the append + * path. In this way, we avoid expending cycles on partitioned rels that + * can't contribute useful pruning information for the problem at hand. + * (It is possible for 'parentrel' to be a child partitioned table, and it + * is also possible for scan-level relations to be child partitioned tables + * rather than leaf partitions. Hence we must construct this relation set + * with reference to the particular append path we're dealing with, rather + * than looking at the full partitioning structure represented in the + * RelOptInfos.) + */ +static List * +add_part_relids(List *allpartrelids, Bitmapset *partrelids) +{ + Index targetpart; + ListCell *lc; + + /* We can easily get the lowest set bit this way: */ + targetpart = bms_next_member(partrelids, -1); + Assert(targetpart > 0); + + /* Look for a matching topmost parent */ + foreach(lc, allpartrelids) + { + Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); + Index currtarget = bms_next_member(currpartrelids, -1); + + if (targetpart == currtarget) + { + /* Found a match, so add any new RT indexes to this hierarchy */ + currpartrelids = bms_add_members(currpartrelids, partrelids); + lfirst(lc) = currpartrelids; + return allpartrelids; + } + } + /* No match, so add the new partition hierarchy to the list */ + return lappend(allpartrelids, partrelids); +} diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index 510145e3c0..3557e07082 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@ -138,7 +138,6 @@ typedef struct PruneStepResult } PruneStepResult; -static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); static List *make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *prunequal, @@ -221,33 +220,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context, * of scan paths for its child rels. * 'prunequal' is a list of potential pruning quals (i.e., restriction * clauses that are applicable to the appendrel). + * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents + * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every + * partition tree involved. */ int make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *subpaths, - List *prunequal) + List *prunequal, + List *allpartrelids) { PartitionPruneInfo *pruneinfo; Bitmapset *allmatchedsubplans = NULL; - List *allpartrelids; List *prunerelinfos; int *relid_subplan_map; ListCell *lc; int i; + Assert(list_length(allpartrelids) > 0); + /* - * Scan the subpaths to see which ones are scans of partition child - * relations, and identify their parent partitioned rels. (Note: we must - * restrict the parent partitioned rels to be parentrel or children of - * parentrel, otherwise we couldn't translate prunequal to match.) - * - * Also construct a temporary array to map from partition-child-relation - * relid to the index in 'subpaths' of the scan plan for that partition. + * Construct a temporary array to map from partition-child-relation relid + * to the index in 'subpaths' of the scan plan for that partition. * (Use of "subplan" rather than "subpath" is a bit of a misnomer, but * we'll let it stand.) For convenience, we use 1-based indexes here, so * that zero can represent an un-filled array entry. */ - allpartrelids = NIL; relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size); i = 1; @@ -256,50 +254,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, Path *path = (Path *) lfirst(lc); RelOptInfo *pathrel = path->parent; - /* We don't consider partitioned joins here */ - if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL) - { - RelOptInfo *prel = pathrel; - Bitmapset *partrelids = NULL; - - /* - * Traverse up to the pathrel's topmost partitioned parent, - * collecting parent relids as we go; but stop if we reach - * parentrel. (Normally, a pathrel's topmost partitioned parent - * is either parentrel or a UNION ALL appendrel child of - * parentrel. But when handling partitionwise joins of - * multi-level partitioning trees, we can see an append path whose - * parentrel is an intermediate partitioned table.) - */ - do - { - AppendRelInfo *appinfo; - - Assert(prel->relid < root->simple_rel_array_size); - appinfo = root->append_rel_array[prel->relid]; - prel = find_base_rel(root, appinfo->parent_relid); - if (!IS_PARTITIONED_REL(prel)) - break; /* reached a non-partitioned parent */ - /* accept this level as an interesting parent */ - partrelids = bms_add_member(partrelids, prel->relid); - if (prel == parentrel) - break; /* don't traverse above parentrel */ - } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL); - - if (partrelids) - { - /* - * Found some relevant parent partitions, which may or may not - * overlap with partition trees we already found. Add new - * information to the allpartrelids list. - */ - allpartrelids = add_part_relids(allpartrelids, partrelids); - /* Also record the subplan in relid_subplan_map[] */ - /* No duplicates please */ - Assert(relid_subplan_map[pathrel->relid] == 0); - relid_subplan_map[pathrel->relid] = i; - } - } + /* No duplicates please */ + Assert(relid_subplan_map[pathrel->relid] == 0); + relid_subplan_map[pathrel->relid] = i; i++; } @@ -368,63 +325,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, return list_length(root->partPruneInfos) - 1; } -/* - * add_part_relids - * Add new info to a list of Bitmapsets of partitioned relids. - * - * Within 'allpartrelids', there is one Bitmapset for each topmost parent - * partitioned rel. Each Bitmapset contains the RT indexes of the topmost - * parent as well as its relevant non-leaf child partitions. Since (by - * construction of the rangetable list) parent partitions must have lower - * RT indexes than their children, we can distinguish the topmost parent - * as being the lowest set bit in the Bitmapset. - * - * 'partrelids' contains the RT indexes of a parent partitioned rel, and - * possibly some non-leaf children, that are newly identified as parents of - * some subpath rel passed to make_partition_pruneinfo(). These are added - * to an appropriate member of 'allpartrelids'. - * - * Note that the list contains only RT indexes of partitioned tables that - * are parents of some scan-level relation appearing in the 'subpaths' that - * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are - * not allowed to be higher than the 'parentrel' associated with the append - * path. In this way, we avoid expending cycles on partitioned rels that - * can't contribute useful pruning information for the problem at hand. - * (It is possible for 'parentrel' to be a child partitioned table, and it - * is also possible for scan-level relations to be child partitioned tables - * rather than leaf partitions. Hence we must construct this relation set - * with reference to the particular append path we're dealing with, rather - * than looking at the full partitioning structure represented in the - * RelOptInfos.) - */ -static List * -add_part_relids(List *allpartrelids, Bitmapset *partrelids) -{ - Index targetpart; - ListCell *lc; - - /* We can easily get the lowest set bit this way: */ - targetpart = bms_next_member(partrelids, -1); - Assert(targetpart > 0); - - /* Look for a matching topmost parent */ - foreach(lc, allpartrelids) - { - Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); - Index currtarget = bms_next_member(currpartrelids, -1); - - if (targetpart == currtarget) - { - /* Found a match, so add any new RT indexes to this hierarchy */ - currpartrelids = bms_add_members(currpartrelids, partrelids); - lfirst(lc) = currpartrelids; - return allpartrelids; - } - } - /* No match, so add the new partition hierarchy to the list */ - return lappend(allpartrelids, partrelids); -} - /* * make_partitionedrel_pruneinfo * Build a List of PartitionedRelPruneInfos, one for each interesting diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 659bd05c0c..a0bb16cff4 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -270,6 +270,13 @@ typedef struct Append List *appendplans; int nasyncplans; /* # of asynchronous plans */ + /* + * RTIs of all partitioned tables whose children are scanned by + * appendplans. The list contains a bitmapset for every partition tree + * covered by this Append. + */ + List *allpartrelids; + /* * All 'appendplans' preceding this index are non-partial plans. All * 'appendplans' from this index onwards are partial plans. @@ -294,6 +301,13 @@ typedef struct MergeAppend List *mergeplans; + /* + * RTIs of all partitioned tables whose children are scanned by + * mergeplans. The list contains a bitmapset for every partition tree + * covered by this MergeAppend. + */ + List *allpartrelids; + /* these fields are just like the sort-key info in struct Sort: */ /* number of sort-key columns */ diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h index a05f91f77d..1621a7319a 100644 --- a/src/include/optimizer/appendinfo.h +++ b/src/include/optimizer/appendinfo.h @@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex, RangeTblEntry *target_rte, Relation target_relation); extern void distribute_row_identity_vars(PlannerInfo *root); +extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids); #endif /* APPENDINFO_H */ diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h index c0d6889d47..2d907d31d4 100644 --- a/src/include/partitioning/partprune.h +++ b/src/include/partitioning/partprune.h @@ -73,7 +73,8 @@ typedef struct PartitionPruneContext extern int make_partition_pruneinfo(struct PlannerInfo *root, struct RelOptInfo *parentrel, List *subpaths, - List *prunequal); + List *prunequal, + List *allpartrelids); extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel); extern Bitmapset *get_matching_partitions(PartitionPruneContext *context, List *pruning_steps); -- 2.35.3 [application/octet-stream] v37-0003-Track-opened-range-table-relations-in-a-List-in-.patch (2.3K, 3-v37-0003-Track-opened-range-table-relations-in-a-List-in-.patch) download | inline diff: From 3c67d3142062334e4ac061f3eb5bc0be306fbb1c Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Mon, 13 Mar 2023 15:59:38 +0900 Subject: [PATCH v37 3/3] Track opened range table relations in a List in EState This makes ExecCloseRangeTableRelations faster when there are many relations in the range table but only a few are opened during execution, such as when run-time pruning kicks in on an Append containing 1000s of partition subplans. --- src/backend/executor/execMain.c | 9 +++++---- src/backend/executor/execUtils.c | 2 ++ src/include/nodes/execnodes.h | 2 ++ 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 0366be9fd6..94f8324cff 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -1630,12 +1630,13 @@ ExecCloseResultRelations(EState *estate) void ExecCloseRangeTableRelations(EState *estate) { - int i; + ListCell *lc; - for (i = 0; i < estate->es_range_table_size; i++) + foreach(lc, estate->es_opened_relations) { - if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + Relation rel = lfirst(lc); + + table_close(rel, NoLock); } } diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index a485e7dfc5..f7053072d9 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -829,6 +829,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) } estate->es_relations[rti - 1] = rel; + estate->es_opened_relations = lappend(estate->es_opened_relations, + rel); } return rel; diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index dfa72848c7..984fd2e423 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState Index es_range_table_size; /* size of the range table arrays */ Relation *es_relations; /* Array of per-range-table-entry Relation * pointers, or NULL if not yet opened */ + List *es_opened_relations; /* List of non-NULL entries in + * es_relations in no specific order */ struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ -- 2.35.3 [application/octet-stream] v37-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch (131.9K, 4-v37-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 4ac824f6f0f6795c3a813d5b046f3b44ee223377 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v37 2/3] Move AcquireExecutorLocks()'s responsibility into the executor This commit introduces a new executor flag EXEC_FLAG_GET_LOCKS that should be passed in eflags to ExecutorStart() if the PlannedStmt comes from a CachedPlan. When set, the executor will take locks on any relations referenced in the plan nodes that need to be initialized for execution. That excludes any partitions that can be pruned during the executor initialization phase, that is, based on the values of only the external (PARAM_EXTERN) parameters. Relations that are not explicitly mentioned in the plan tree, such as views and non-leaf partition parents whose children are mentioned in Append/MergeAppend nodes, are locked separately. After taking each lock, the executor calls CachedPlanStillValid() to check if CachedPlan.is_valid has been reset by PlanCacheRelCallback() due to concurrent modification of relations referenced in the plan. If it is found that the CachedPlan is indeed invalid, the recursive ExecInitNode() traversal is aborted at that point. To allow the proper cleanup of such a partially initialized planstate tree, ExecEndNode() subroutines of various plan nodes have been fixed to account for potentially uninitialized fields. It is the caller's (of ExecutorStart()) responsibility to call ExecutorEnd() even on a QueryDesc containing such a partially initialized PlanState tree. Call sites that use plancache (GetCachedPlan) to get the plan trees to pass to the executor for execution should now be prepared to handle the case that the plan tree may be flagged by the executor as stale as described above. To that end, this commit refactors the relevant code sites to move the ExecutorStart() call closer to the GetCachedPlan() call to reduce the friction in the cases where replanning is needed due to a CachedPlan being marked stale in this manner. Callers must check that QueryDesc.plan_valid is true before passing it on to ExecutorRun() for execution. PortalStart() now performs CreateQueryDesc() and ExecutorStart() for all portal strategies, including those pertaining to multiple queries. The QueryDescs for strategies handled by PortalRunMulti() are remembered in the Portal in a new List field 'qdescs', allocated in a new memory context 'queryContext'. This new arrangment is to make it easier to discard and recreate a Portal if the CachedPlan goes stale during setup. --- contrib/postgres_fdw/postgres_fdw.c | 4 + src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 150 ++++++--- src/backend/commands/extension.c | 2 + src/backend/commands/matview.c | 3 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 32 +- src/backend/executor/execMain.c | 89 ++++- src/backend/executor/execParallel.c | 8 +- src/backend/executor/execPartition.c | 15 + src/backend/executor/execProcnode.c | 9 + src/backend/executor/execUtils.c | 60 +++- src/backend/executor/functions.c | 2 + src/backend/executor/nodeAgg.c | 23 +- src/backend/executor/nodeAppend.c | 23 +- src/backend/executor/nodeBitmapAnd.c | 10 +- src/backend/executor/nodeBitmapHeapscan.c | 10 +- src/backend/executor/nodeBitmapIndexscan.c | 2 + src/backend/executor/nodeBitmapOr.c | 10 +- src/backend/executor/nodeCtescan.c | 6 +- src/backend/executor/nodeCustom.c | 12 +- src/backend/executor/nodeForeignscan.c | 22 +- src/backend/executor/nodeFunctionscan.c | 3 +- src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 5 +- src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 13 +- src/backend/executor/nodeIncrementalSort.c | 14 +- src/backend/executor/nodeIndexonlyscan.c | 7 +- src/backend/executor/nodeIndexscan.c | 7 +- src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 5 +- src/backend/executor/nodeMemoize.c | 12 +- src/backend/executor/nodeMergeAppend.c | 23 +- src/backend/executor/nodeMergejoin.c | 10 +- src/backend/executor/nodeModifyTable.c | 13 +- .../executor/nodeNamedtuplestorescan.c | 3 +- src/backend/executor/nodeNestloop.c | 7 +- src/backend/executor/nodeProjectSet.c | 5 +- src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 5 +- src/backend/executor/nodeSamplescan.c | 5 +- src/backend/executor/nodeSeqscan.c | 5 +- src/backend/executor/nodeSetOp.c | 5 +- src/backend/executor/nodeSort.c | 8 +- src/backend/executor/nodeSubqueryscan.c | 5 +- src/backend/executor/nodeTableFuncscan.c | 3 +- src/backend/executor/nodeTidrangescan.c | 5 +- src/backend/executor/nodeTidscan.c | 5 +- src/backend/executor/nodeUnique.c | 5 +- src/backend/executor/nodeValuesscan.c | 3 +- src/backend/executor/nodeWindowAgg.c | 55 +++- src/backend/executor/nodeWorktablescan.c | 3 +- src/backend/executor/spi.c | 53 ++- src/backend/nodes/outfuncs.c | 1 + src/backend/nodes/readfuncs.c | 1 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 13 +- src/backend/tcop/pquery.c | 311 ++++++++++-------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 ++------ src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 16 + src/include/nodes/execnodes.h | 2 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 14 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 117 +++++++ .../specs/cached-plan-replan.spec | 50 +++ 82 files changed, 1230 insertions(+), 424 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index beea1ac687..e9f77d5711 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 878d2fd172..826a47af0a 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -415,12 +416,95 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (es->generic) + eflags |= EXEC_FLAG_EXPLAIN_GENERIC; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -524,29 +608,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - Assert(plannedstmt->commandType != CMD_UTILITY); - - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -555,40 +626,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (es->generic) - eflags |= EXEC_FLAG_EXPLAIN_GENERIC; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4862,6 +4899,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index 0eabe18335..5a76343123 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -797,11 +797,13 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); ExecutorStart(qdesc, 0); + Assert(qdesc->plan_valid); ExecutorRun(qdesc, ForwardScanDirection, 0, true); ExecutorFinish(qdesc); ExecutorEnd(qdesc); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index c00b9df3e3..80f2c38b35 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); /* call ExecutorStart to prepare the plan for execution */ ExecutorStart(queryDesc, 0); + Assert(queryDesc->plan_valid); /* run the plan */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..c9070ed97f 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, it + * must be recreated if portal->plan_valid is false which tells that the + * cached plan was found to have been invalidated when initializing one of + * the plan trees contained in it. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 1b007dc32c..0366be9fd6 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,32 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan, in which case only + * some of the relations referenced in the plan would have been locked; to + * wit, those that AcquirePlannerLocks() deems necessary. Locks necessary + * to fully validate such a plan tree, including relations that are added by + * the planner, will be taken when initializing the plan tree in InitPlan(); + * the the caller must have set the EXEC_FLAG_GET_LOCKS bit in eflags. If the + * CachedPlan gets invalidated as these locks are taken, plan tree + * initialization is suspended at the point when such invalidation is first + * detected and InitPlan() returns after setting queryDesc->plan_valid to + * false. queryDesc->planstate would be pointing to a potentially partially + * initialized PlanState tree in that case. Callers must retry the execution + * with a freshly created CachedPlan in that case, after properly freeing the + * partially valid QueryDesc. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + (CachedPlanStillValid(queryDesc->cplan) && + (eflags & EXEC_FLAG_GET_LOCKS) != 0)); + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +603,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +816,19 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } - /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -801,20 +839,32 @@ InitPlan(QueryDesc *queryDesc, int eflags) Plan *plan = plannedstmt->planTree; List *rangeTable = plannedstmt->rtable; EState *estate = queryDesc->estate; - PlanState *planstate; + PlanState *planstate = NULL; TupleDesc tupType; ListCell *l; int i; /* - * Do permissions checks + * Set up range table in EState. */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + + /* Make sure ExecPlanStillValid() can work. */ + estate->es_cachedplan = queryDesc->cplan; /* - * initialize the node's execution state + * Lock any views that were mentioned in the query if needed. View + * relations must be locked separately like this, because they are not + * referenced in the plan tree. */ - ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + ExecLockViewRelations(plannedstmt->viewRelations, estate); + if (!ExecPlanStillValid(estate)) + goto failed; + + /* + * Do permissions checks + */ + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); estate->es_plannedstmt = plannedstmt; estate->es_part_prune_infos = plannedstmt->partPruneInfos; @@ -849,6 +899,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -919,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); + if (!ExecPlanStillValid(estate)) + goto failed; i++; } @@ -929,6 +983,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -972,6 +1028,17 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such. Note that we do + * set planstate, even if it may only be partially initialized, so that + * ExecEndPlan() can process it. + */ + queryDesc->planstate = planstate; + queryDesc->plan_valid = false; } /* @@ -1389,7 +1456,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -2797,7 +2864,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2884,6 +2952,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } @@ -2937,6 +3006,10 @@ EvalPlanQualEnd(EPQState *epqstate) MemoryContext oldcontext; ListCell *l; + /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */ + if (epqstate->parentestate == NULL) + return; + rtsize = epqstate->parentestate->es_range_table_size; /* diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..df4cc5ddaf 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); @@ -1432,6 +1437,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc) /* Start up the executor */ queryDesc->plannedstmt->jitFlags = fpes->jit_flags; ExecutorStart(queryDesc, fpes->eflags); + Assert(queryDesc->plan_valid); /* Special executor initialization steps for parallel workers */ queryDesc->planstate->state->es_query_dsa = area; diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 9799968a42..3425ffcca7 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -513,6 +513,12 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate, oldcxt = MemoryContextSwitchTo(proute->memcxt); + /* + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. + */ partrel = table_open(partOid, RowExclusiveLock); leaf_part_rri = makeNode(ResultRelInfo); @@ -1111,6 +1117,11 @@ ExecInitPartitionDispatchInfo(EState *estate, * Only sub-partitioned tables need to be locked here. The root * partitioned table will already have been locked as it's referenced in * the query's rtable. + * + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. */ if (partoid != RelationGetRelid(proute->partition_root)) rel = table_open(partoid, RowExclusiveLock); @@ -1817,6 +1828,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1943,6 +1956,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..6f3c37b6fd 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return result; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -403,6 +406,12 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); subps = lappend(subps, sstate); + if (!ExecPlanStillValid(estate)) + { + /* Don't lose track of those initialized. */ + result->initPlan = subps; + return result; + } } result->initPlan = subps; diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 012dbb6965..a485e7dfc5 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -804,7 +804,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -833,6 +834,61 @@ ExecGetRangeTableRelation(EState *estate, Index rti) return rel; } +/* + * ExecLockViewRelations + * Lock view relations, if any, in a given query + */ +void +ExecLockViewRelations(List *viewRelations, EState *estate) +{ + ListCell *lc; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + LockRelationOid(rte->relid, rte->rellockmode); + } +} + +/* + * ExecLockAppendNonLeafRelations + * Lock non-leaf relations whose children are scanned by a given + * Append/MergeAppend node + */ +void +ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids) +{ + ListCell *l; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(l, allpartrelids) + { + Bitmapset *partrelids = lfirst_node(Bitmapset, l); + int i; + + i = -1; + while ((i = bms_next_member(partrelids, i)) > 0) + { + RangeTblEntry *rte = exec_rt_fetch(i, estate); + + LockRelationOid(rte->relid, rte->rellockmode); + } + } +} + /* * ExecInitResultRelation * Open relation given by the passed-in RT index and fill its @@ -848,6 +904,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index f55424eb5a..c88f72bc4e 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, @@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) else eflags = 0; /* default run-to-completion flags */ ExecutorStart(es->qd, eflags); + Assert(es->qd->plan_valid); } es->status = F_EXEC_RUN; diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 19342a420c..06e0d7d149 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3134,15 +3134,18 @@ hashagg_reset_spill_state(AggState *aggstate) { HashAggSpill *spill = &aggstate->hash_spills[setno]; - pfree(spill->ntuples); - pfree(spill->partitions); + if (spill->ntuples) + pfree(spill->ntuples); + if (spill->partitions) + pfree(spill->partitions); } pfree(aggstate->hash_spills); aggstate->hash_spills = NULL; } /* free batches */ - list_free_deep(aggstate->hash_batches); + if (aggstate->hash_batches) + list_free_deep(aggstate->hash_batches); aggstate->hash_batches = NIL; /* close tape set */ @@ -3296,6 +3299,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return aggstate; /* * initialize source tuple type. @@ -4336,10 +4341,13 @@ ExecEndAgg(AggState *node) { AggStatePerTrans pertrans = &node->pertrans[transno]; - for (setno = 0; setno < numGroupingSets; setno++) + if (pertrans) { - if (pertrans->sortstates[setno]) - tuplesort_end(pertrans->sortstates[setno]); + for (setno = 0; setno < numGroupingSets; setno++) + { + if (pertrans->sortstates[setno]) + tuplesort_end(pertrans->sortstates[setno]); + } } } @@ -4357,7 +4365,8 @@ ExecEndAgg(AggState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index c185b11c67..091f979c46 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -109,10 +109,11 @@ AppendState * ExecInitAppend(Append *node, EState *estate, int eflags) { AppendState *appendstate = makeNode(AppendState); - PlanState **appendplanstates; + PlanState **appendplanstates = NULL; Bitmapset *validsubplans; Bitmapset *asyncplans; int nplans; + int ninited = 0; int nasyncplans; int firstvalid; int i, @@ -133,6 +134,15 @@ ExecInitAppend(Append *node, EState *estate, int eflags) appendstate->as_syncdone = false; appendstate->as_begun = false; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -148,6 +158,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + goto early_exit; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -222,11 +234,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } appendstate->as_first_partial_plan = firstvalid; - appendstate->appendplans = appendplanstates; - appendstate->as_nplans = nplans; /* Initialize async state */ appendstate->as_asyncplans = asyncplans; @@ -276,6 +289,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags) /* For parallel query, this will be overridden later. */ appendstate->choose_next_subplan = choose_next_subplan_locally; +early_exit: + appendstate->appendplans = appendplanstates; + appendstate->as_nplans = ninited; + return appendstate; } diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..acc6c50e20 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -57,6 +57,7 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) BitmapAndState *bitmapandstate = makeNode(BitmapAndState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -77,8 +78,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) bitmapandstate->ps.plan = (Plan *) node; bitmapandstate->ps.state = estate; bitmapandstate->ps.ExecProcNode = ExecBitmapAnd; - bitmapandstate->bitmapplans = bitmapplanstates; - bitmapandstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -89,6 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -99,6 +101,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmapandstate->bitmapplans = bitmapplanstates; + bitmapandstate->nplans = ninited; + return bitmapandstate; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..e6a689eefb 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -665,7 +665,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subplans @@ -693,7 +694,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) /* * close heap scan */ - table_endscan(scanDesc); + if (scanDesc) + table_endscan(scanDesc); } /* ---------------------------------------------------------------- @@ -763,11 +765,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c index 83ec9ede89..cc8332ef68 100644 --- a/src/backend/executor/nodeBitmapIndexscan.c +++ b/src/backend/executor/nodeBitmapIndexscan.c @@ -263,6 +263,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->biss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..babad1b4b2 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -58,6 +58,7 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) BitmapOrState *bitmaporstate = makeNode(BitmapOrState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -78,8 +79,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) bitmaporstate->ps.plan = (Plan *) node; bitmaporstate->ps.state = estate; bitmaporstate->ps.ExecProcNode = ExecBitmapOr; - bitmaporstate->bitmapplans = bitmapplanstates; - bitmaporstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -90,6 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -100,6 +102,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmaporstate->bitmapplans = bitmapplanstates; + bitmaporstate->nplans = ninited; + return bitmaporstate; } diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c index cc4c4243e2..eed5b75a4f 100644 --- a/src/backend/executor/nodeCtescan.c +++ b/src/backend/executor/nodeCtescan.c @@ -297,14 +297,16 @@ ExecEndCteScan(CteScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * If I am the leader, free the tuplestore. */ if (node->leader == node) { - tuplestore_end(node->cte_table); + if (node->cte_table) + tuplestore_end(node->cte_table); node->cte_table = NULL; } } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..b03499fae5 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return css; css->ss.ss_currentRelation = scan_rel; } @@ -127,6 +129,10 @@ ExecCustomScan(PlanState *pstate) void ExecEndCustomScan(CustomScanState *node) { + /* + * XXX - BeginCustomScan() may not have occurred if ExecInitCustomScan() + * hit the early exit case. + */ Assert(node->methods->EndCustomScan != NULL); node->methods->EndCustomScan(node); @@ -134,8 +140,10 @@ ExecEndCustomScan(CustomScanState *node) ExecFreeExprContext(&node->ss.ps); /* Clean out the tuple table */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } void diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..d3f0a65485 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * Tell the FDW to initialize the scan. @@ -300,14 +304,17 @@ ExecEndForeignScan(ForeignScanState *node) ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; EState *estate = node->ss.ps.state; - /* Let the FDW shut down */ - if (plan->operation != CMD_SELECT) + /* Let the FDW shut down if needed. */ + if (node->fdw_state) { - if (estate->es_epq_active == NULL) - node->fdwroutine->EndDirectModify(node); + if (plan->operation != CMD_SELECT) + { + if (estate->es_epq_active == NULL) + node->fdwroutine->EndDirectModify(node); + } + else + node->fdwroutine->EndForeignScan(node); } - else - node->fdwroutine->EndForeignScan(node); /* Shut down any outer plan. */ if (outerPlanState(node)) @@ -319,7 +326,8 @@ ExecEndForeignScan(ForeignScanState *node) /* clean out the tuple table */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c index dd06ef8aee..792ecda4a9 100644 --- a/src/backend/executor/nodeFunctionscan.c +++ b/src/backend/executor/nodeFunctionscan.c @@ -533,7 +533,8 @@ ExecEndFunctionScan(FunctionScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release slots and tuplestore resources diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..365d3af3e4 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gatherstate; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..8d2809f079 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gm_state; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..e0832bb778 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return grpstate; /* * Initialize scan slot and type. @@ -231,7 +233,8 @@ ExecEndGroup(GroupState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index 748c9b0024..891bcee919 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return hashstate; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index f189fb4d28..93ce0c8be0 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -691,8 +691,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* @@ -813,9 +817,12 @@ ExecEndHashJoin(HashJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->hj_OuterTupleSlot); - ExecClearTuple(node->hj_HashTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->hj_OuterTupleSlot) + ExecClearTuple(node->hj_OuterTupleSlot); + if (node->hj_HashTupleSlot) + ExecClearTuple(node->hj_HashTupleSlot); /* * clean up subtrees diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..6b2da56044 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return incrsortstate; /* * Initialize scan slot and type. @@ -1080,12 +1082,16 @@ ExecEndIncrementalSort(IncrementalSortState *node) SO_printf("ExecEndIncrementalSort: shutting down sort node\n"); /* clean out the scan tuple */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* must drop standalone tuple slots from outer node */ - ExecDropSingleTupleTableSlot(node->group_pivot); - ExecDropSingleTupleTableSlot(node->transfer_tuple); + if (node->group_pivot) + ExecDropSingleTupleTableSlot(node->group_pivot); + if (node->transfer_tuple) + ExecDropSingleTupleTableSlot(node->transfer_tuple); /* * Release tuplesort resources. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..b60a086464 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -394,7 +394,8 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if(node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -565,6 +568,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..628c233919 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -808,7 +808,8 @@ ExecEndIndexScan(IndexScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -970,6 +973,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->iss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..2fcbde74ed 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return limitstate; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..3a8aa2b5a4 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return lrstate; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..f146ebb1d7 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return matstate; /* * Initialize result type and slot. No need to initialize projection info @@ -242,7 +244,8 @@ ExecEndMaterial(MaterialState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 4f04269e26..3003ee1e5c 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mstate; /* * Initialize return slot and type. No need to initialize projection info @@ -1043,6 +1045,7 @@ ExecEndMemoize(MemoizeState *node) { #ifdef USE_ASSERT_CHECKING /* Validate the memory accounting code is correct in assert builds. */ + if (node->hashtable) { int count; uint64 mem = 0; @@ -1089,11 +1092,14 @@ ExecEndMemoize(MemoizeState *node) } /* Remove the cache context */ - MemoryContextDelete(node->tableContext); + if (node->tableContext) + MemoryContextDelete(node->tableContext); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to cache result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * free exprcontext diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..40bba35499 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -65,9 +65,10 @@ MergeAppendState * ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) { MergeAppendState *mergestate = makeNode(MergeAppendState); - PlanState **mergeplanstates; + PlanState **mergeplanstates = NULL; Bitmapset *validsubplans; int nplans; + int ninited = 0; int i, j; @@ -81,6 +82,15 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) mergestate->ps.state = estate; mergestate->ps.ExecProcNode = ExecMergeAppend; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -96,6 +106,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return mergestate; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -122,8 +134,6 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) } mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *)); - mergestate->mergeplans = mergeplanstates; - mergestate->ms_nplans = nplans; mergestate->ms_slots = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans); mergestate->ms_heap = binaryheap_allocate(nplans, heap_compare_slots, @@ -152,6 +162,9 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } mergestate->ps.ps_ProjInfo = NULL; @@ -188,6 +201,10 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) */ mergestate->ms_initialized = false; +early_exit: + mergestate->mergeplans = mergeplanstates; + mergestate->ms_nplans = ninited; + return mergestate; } diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..968be05568 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return mergestate; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* @@ -1642,8 +1646,10 @@ ExecEndMergeJoin(MergeJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->mj_MarkedTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->mj_MarkedTupleSlot) + ExecClearTuple(node->mj_MarkedTupleSlot); /* * shut down the subplans diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index e350375681..8a70543326 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -3900,6 +3900,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) Plan *subplan = outerPlan(node); CmdType operation = node->operation; int nrels = list_length(node->resultRelations); + int ninited = 0; ResultRelInfo *resultRelInfo; List *arowmarks; ListCell *l; @@ -3921,7 +3922,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) mtstate->canSetTag = node->canSetTag; mtstate->mt_done = false; - mtstate->mt_nrels = nrels; mtstate->resultRelInfo = (ResultRelInfo *) palloc(nrels * sizeof(ResultRelInfo)); @@ -3956,6 +3956,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -3982,6 +3985,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * For child result relations, store the root result relation @@ -4009,11 +4014,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * Do additional per-result-relation initialization. */ - for (i = 0; i < nrels; i++) + for (i = 0; i < nrels; i++, ninited++) { resultRelInfo = &mtstate->resultRelInfo[i]; @@ -4362,6 +4369,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) estate->es_auxmodifytables = lcons(mtstate, estate->es_auxmodifytables); +early_exit: + mtstate->mt_nrels = ninited; return mtstate; } diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c index 46832ad82f..1f92c43d3b 100644 --- a/src/backend/executor/nodeNamedtuplestorescan.c +++ b/src/backend/executor/nodeNamedtuplestorescan.c @@ -174,7 +174,8 @@ ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..deda0c2559 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; /* * Initialize result slot, type and projection. @@ -372,7 +376,8 @@ ExecEndNestLoop(NestLoopState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); /* * close down subplans diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..85d20c4680 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return state; /* * we don't use inner plan @@ -328,7 +330,8 @@ ExecEndProjectSet(ProjectSetState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..967fe4f287 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..c549b684a3 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return resstate; /* * we don't use inner plan @@ -248,7 +250,8 @@ ExecEndResult(ResultState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..b3bc9b1f77 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; @@ -198,7 +200,8 @@ ExecEndSampleScan(SampleScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..e7ca19ee4e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, @@ -200,7 +202,8 @@ ExecEndSeqScan(SeqScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..95950a5c20 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return setopstate; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* @@ -583,7 +585,8 @@ void ExecEndSetOp(SetOpState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* free subsidiary stuff including hashtable */ if (node->tableContext) diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..89fef86aba 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return sortstate; /* * Initialize scan slot and type. @@ -306,9 +308,11 @@ ExecEndSort(SortState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * Release tuplesort resources diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..9b8cddc89f 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return subquerystate; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) @@ -177,7 +179,8 @@ ExecEndSubqueryScan(SubqueryScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subquery diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c index 0c6c912778..d7536953f1 100644 --- a/src/backend/executor/nodeTableFuncscan.c +++ b/src/backend/executor/nodeTableFuncscan.c @@ -223,7 +223,8 @@ ExecEndTableFuncScan(TableFuncScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..1ae451d7a6 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -342,7 +342,8 @@ ExecEndTidRangeScan(TidRangeScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -386,6 +387,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidrangestate; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 862bd0330b..9fe76b1c60 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -483,7 +483,8 @@ ExecEndTidScan(TidScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -529,6 +530,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidstate; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..69f23b02c6 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return uniquestate; /* * Initialize result slot and type. Unique nodes do no projections, so @@ -169,7 +171,8 @@ void ExecEndUnique(UniqueState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); ExecFreeExprContext(&node->ps); diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c index 32ace63017..f5dedbab63 100644 --- a/src/backend/executor/nodeValuesscan.c +++ b/src/backend/executor/nodeValuesscan.c @@ -340,7 +340,8 @@ ExecEndValuesScan(ValuesScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 7c07fb0684..616bb97675 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -1334,7 +1334,7 @@ release_partition(WindowAggState *winstate) WindowStatePerFunc perfuncstate = &(winstate->perfunc[i]); /* Release any partition-local state of this window function */ - if (perfuncstate->winobj) + if (perfuncstate && perfuncstate->winobj) perfuncstate->winobj->localmem = NULL; } @@ -1344,12 +1344,17 @@ release_partition(WindowAggState *winstate) * any aggregate temp data). We don't rely on retail pfree because some * aggregates might have allocated data we don't have direct pointers to. */ - MemoryContextResetAndDeleteChildren(winstate->partcontext); - MemoryContextResetAndDeleteChildren(winstate->aggcontext); - for (i = 0; i < winstate->numaggs; i++) + if (winstate->partcontext) + MemoryContextResetAndDeleteChildren(winstate->partcontext); + if (winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->aggcontext); + if (winstate->peragg) { - if (winstate->peragg[i].aggcontext != winstate->aggcontext) - MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + for (i = 0; i < winstate->numaggs; i++) + { + if (winstate->peragg[i].aggcontext != winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + } } if (winstate->buffer) @@ -2451,6 +2456,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return winstate; /* * initialize source tuple type (which is also the tuple type that we'll @@ -2679,11 +2686,16 @@ ExecEndWindowAgg(WindowAggState *node) release_partition(node); - ExecClearTuple(node->ss.ss_ScanTupleSlot); - ExecClearTuple(node->first_part_slot); - ExecClearTuple(node->agg_row_slot); - ExecClearTuple(node->temp_slot_1); - ExecClearTuple(node->temp_slot_2); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->first_part_slot) + ExecClearTuple(node->first_part_slot); + if (node->agg_row_slot) + ExecClearTuple(node->agg_row_slot); + if (node->temp_slot_1) + ExecClearTuple(node->temp_slot_1); + if (node->temp_slot_2) + ExecClearTuple(node->temp_slot_2); if (node->framehead_slot) ExecClearTuple(node->framehead_slot); if (node->frametail_slot) @@ -2696,16 +2708,23 @@ ExecEndWindowAgg(WindowAggState *node) node->ss.ps.ps_ExprContext = node->tmpcontext; ExecFreeExprContext(&node->ss.ps); - for (i = 0; i < node->numaggs; i++) + if (node->peragg) { - if (node->peragg[i].aggcontext != node->aggcontext) - MemoryContextDelete(node->peragg[i].aggcontext); + for (i = 0; i < node->numaggs; i++) + { + if (node->peragg[i].aggcontext != node->aggcontext) + MemoryContextDelete(node->peragg[i].aggcontext); + } } - MemoryContextDelete(node->partcontext); - MemoryContextDelete(node->aggcontext); + if (node->partcontext) + MemoryContextDelete(node->partcontext); + if (node->aggcontext) + MemoryContextDelete(node->aggcontext); - pfree(node->perfunc); - pfree(node->peragg); + if (node->perfunc) + pfree(node->perfunc); + if (node->peragg) + pfree(node->peragg); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c index 0c13448236..d70c6afde3 100644 --- a/src/backend/executor/nodeWorktablescan.c +++ b/src/backend/executor/nodeWorktablescan.c @@ -200,7 +200,8 @@ ExecEndWorkTableScan(WorkTableScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index e3a170c38b..26a9ea342a 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2668,14 +2680,36 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + + /* Take locks if using a CachedPlan */ + if (qdesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + Assert(cplan); + ReleaseCachedPlan(cplan, plan_owner); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2850,10 +2884,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2897,14 +2930,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index ba00b99249..955286513d 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -513,6 +513,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ WRITE_OID_FIELD(relid); + WRITE_CHAR_FIELD(relkind); WRITE_INT_FIELD(rellockmode); WRITE_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index 597e5b3ea8..a136ae1d60 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -503,6 +503,7 @@ _readRangeTblEntry(void) READ_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ READ_OID_FIELD(relid); + READ_CHAR_FIELD(relkind); READ_INT_FIELD(rellockmode); READ_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 62b3ec96cc..5f3ffd98af 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 5cc8366af6..f13240bf33 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -604,6 +605,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index 980dc1816f..1631c8b993 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1849,11 +1849,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..6d0ea07801 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1199,6 +1199,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1703,6 +1704,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1994,10 +1996,19 @@ exec_bind_message(StringInfo input_message) PopActiveSnapshot(); /* - * And we're ready to start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..c93a950d7f 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt, qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +353,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +419,56 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti(). */ + if (portal->strategy == PORTAL_ONE_RETURNING || + portal->strategy == PORTAL_ONE_MOD_WITH) + portal->qdescs = list_make1(queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +476,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +500,90 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool first = true; + + /* Take locks if using a CachedPlan */ + myeflags = 0; + if (portal->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* + * Push the snapshot to be used by the executor. + */ + if (!is_utility) + { + /* + * Must copy the snapshot if we'll need to update + * its command ID. + */ + if (!first) + PushCopiedSnapshot(GetTransactionSnapshot()); + else + PushActiveSnapshot(GetTransactionSnapshot()); + } + + /* + * From the 2nd statement onwards, update the command + * ID and the snapshot to match. + */ + if (!first) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + first = false; + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + !is_utility ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + if (is_utility) + continue; + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + ExecutorStart(queryDesc, myeflags); + PopActiveSnapshot(); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + portal->plan_valid = false; + goto early_exit; + } + } + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +595,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1193,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1214,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1271,23 +1272,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1362,15 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c7607895cd..014cd476f4 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 3d3e632a0c..392abb5150 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index f9e6bf3d4a..a6ac772400 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -61,6 +62,10 @@ * WITH_NO_DATA indicates that we are performing REFRESH MATERIALIZED VIEW * ... WITH NO DATA. Currently, the only effect is to suppress errors about * scanning unpopulated materialized views. + * + * GET_LOCKS indicates that the caller of ExecutorStart() is executing a + * cached plan which must be validated by taking the remaining locks necessary + * for execution. */ #define EXEC_FLAG_EXPLAIN_ONLY 0x0001 /* EXPLAIN, no ANALYZE */ #define EXEC_FLAG_EXPLAIN_GENERIC 0x0002 /* EXPLAIN (GENERIC_PLAN) */ @@ -69,6 +74,8 @@ #define EXEC_FLAG_MARK 0x0010 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0020 /* skip AfterTrigger setup */ #define EXEC_FLAG_WITH_NO_DATA 0x0040 /* REFRESH ... WITH NO DATA */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should the executor lock + * relations? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -255,6 +262,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode @@ -589,6 +603,8 @@ exec_rt_fetch(Index rti, EState *estate) } extern Relation ExecGetRangeTableRelation(EState *estate, Index rti); +extern void ExecLockViewRelations(List *viewRelations, EState *estate); +extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids); extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Index rti); diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index d97f5a8e7d..dfa72848c7 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -623,6 +623,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d61a62da19..9b888b0d75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index a0bb16cff4..7cae624bbd 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..8990fe72e3 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor on every relation lock taken when initializing the + * plan tree in the CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..4f450b9d9b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,117 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +-------------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo11 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo11_a_idx + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +----------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo11 foo_1 + Filter: (a = $1) +(4 rows) + + +starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock +step s1prep2: SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Index Only Scan using foo11_a_idx on foo11 t1 + -> Materialize + -> Index Scan using foo11_a_idx on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(18 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec2: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec2: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +--------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Sort + Sort Key: t1.a + -> Seq Scan on foo11 t1 + -> Sort + Sort Key: t2.a + -> Seq Scan on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(21 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..67cfed7044 --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,50 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a); + CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1); + CREATE INDEX foo11_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +step "s1prep2" { SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +step "s1exec2" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo11_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" +permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock" -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-03-27 14:00 Amit Langote <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Amit Langote @ 2023-03-27 14:00 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]>; Tom Lane <[email protected]> > > On Tue, Mar 14, 2023 at 7:07 PM Amit Langote <[email protected]> wrote: > > > * I decided to initialize QueryDesc.planstate even in the cases where > > > ExecInitNode() traversal is aborted in the middle on detecting > > > CachedPlan invalidation such that it points to a partially initialized > > > PlanState tree. My earlier thinking that each PlanState node need not > > > be visited for resource cleanup in such cases was naive after all. To > > > that end, I've fixed the ExecEndNode() subroutines of all Plan node > > > types to account for potentially uninitialized fields. There are a > > > couple of cases where I'm a bit doubtful though. In > > > ExecEndCustomScan(), there's no indication in CustomScanState whether > > > it's OK to call EndCustomScan() when BeginCustomScan() may not have > > > been called. For ForeignScanState, I've assumed that > > > ForeignScanState.fdw_state being set can be used as a marker that > > > BeginForeignScan would have been called, though maybe that's not a > > > solid assumption. It seems I hadn't noted in the ExecEndNode()'s comment that all node types' recursive subroutines need to handle the change made by this patch that the corresponding ExecInitNode() subroutine may now return early without having initialized all state struct fields. Also noted in the documentation for CustomScan and ForeignScan that the Begin*Scan callback may not have been called at all, so the End*Scan should handle that gracefully. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com Attachments: [application/octet-stream] v38-0001-Add-field-to-store-partitioned-relids-to-Append-.patch (20.2K, 2-v38-0001-Add-field-to-store-partitioned-relids-to-Append-.patch) download | inline diff: From dfc41510ef3ebec38e7a56b639ffa41193109b43 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Thu, 9 Mar 2023 11:26:06 +0900 Subject: [PATCH v38 1/3] Add field to store partitioned relids to Append/MergeAppend A future commit would like to move the timing of locking relations referenced in a cached plan to ExecInitNode() traversal of the plan tree from the current loop-over-rangetable in AcquireExecutorLocks(). Given that partitioned tables (their RT indexes) would not be accessible via the new way of finding the relations to lock, add a field to Append/MergeAppend to track them separately. This refactors the code to look up partitioned parent relids from a given list of leaf partition subpaths of an Append/MergeAppend out of make_partition_pruneinfo() into its own function called add_append_subpath_partrelids(). Though, the code needs to be generalized to the cases where child rels can be joinrels or upper (grouping) rels. Also, to make it easier to traverse the parent chain of a child grouping rel, this makes its RelOptInfo.parent to be set, which is already done for baserels and joinrels. --- src/backend/optimizer/plan/createplan.c | 36 +++++-- src/backend/optimizer/plan/planner.c | 3 + src/backend/optimizer/util/appendinfo.c | 134 ++++++++++++++++++++++++ src/backend/partitioning/partprune.c | 124 +++------------------- src/include/nodes/plannodes.h | 14 +++ src/include/optimizer/appendinfo.h | 3 + src/include/partitioning/partprune.h | 3 +- 7 files changed, 194 insertions(+), 123 deletions(-) diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index 910ffbf1e1..794cdb5e3b 100644 --- a/src/backend/optimizer/plan/createplan.c +++ b/src/backend/optimizer/plan/createplan.c @@ -25,6 +25,7 @@ #include "nodes/extensible.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/appendinfo.h" #include "optimizer/clauses.h" #include "optimizer/cost.h" #include "optimizer/optimizer.h" @@ -1209,6 +1210,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) Oid *nodeCollations = NULL; bool *nodeNullsFirst = NULL; bool consider_async = false; + List *allpartrelids = NIL; /* * The subpaths list could be empty, if every child was proven empty by @@ -1350,18 +1352,24 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) ++nasyncplans; } + /* Populate partitioned parent relids. */ + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + plan->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ plan->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1381,7 +1389,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags) if (prunequal != NIL) plan->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } plan->appendplans = subplans; @@ -1425,6 +1434,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, List *subplans = NIL; ListCell *subpaths; RelOptInfo *rel = best_path->path.parent; + List *allpartrelids = NIL; /* * We don't have the actual creation of the MergeAppend node split out @@ -1514,18 +1524,23 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, subplan = (Plan *) sort; } + allpartrelids = add_append_subpath_partrelids(root, subpath, rel, + allpartrelids); + subplans = lappend(subplans, subplan); } + node->allpartrelids = allpartrelids; + /* Set below if we find quals that we can use to run-time prune */ node->part_prune_index = -1; /* - * If any quals exist, they may be useful to perform further partition - * pruning during execution. Gather information needed by the executor to - * do partition pruning. + * If scanning partitions, check if there are quals that may be useful to + * perform further partition pruning during execution. Gather information + * needed by the executor to do partition pruning. */ - if (enable_partition_pruning) + if (enable_partition_pruning && allpartrelids != NIL) { List *prunequal; @@ -1537,7 +1552,8 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path, if (prunequal != NIL) node->part_prune_index = make_partition_pruneinfo(root, rel, best_path->subpaths, - prunequal); + prunequal, + allpartrelids); } node->mergeplans = subplans; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index a1873ce26d..62b3ec96cc 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -7801,8 +7801,11 @@ create_partitionwise_grouping_paths(PlannerInfo *root, agg_costs, gd, &child_extra, &child_partially_grouped_rel); + /* Mark as child of grouped_rel. */ + child_grouped_rel->parent = grouped_rel; if (child_partially_grouped_rel) { + child_partially_grouped_rel->parent = grouped_rel; partially_grouped_live_children = lappend(partially_grouped_live_children, child_partially_grouped_rel); diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c index 9d377385f1..4876742ab2 100644 --- a/src/backend/optimizer/util/appendinfo.c +++ b/src/backend/optimizer/util/appendinfo.c @@ -40,6 +40,7 @@ static void make_inh_translation_list(Relation oldrelation, AppendRelInfo *appinfo); static Node *adjust_appendrel_attrs_mutator(Node *node, adjust_appendrel_attrs_context *context); +static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); /* @@ -1031,3 +1032,136 @@ distribute_row_identity_vars(PlannerInfo *root) } } } + +/* + * add_append_subpath_partrelids + * Look up a child subpath's rel's partitioned parent relids up to + * parentrel and add the bitmapset containing those into + * 'allpartrelids' + */ +List * +add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids) +{ + RelOptInfo *prel = subpath->parent; + Relids partrelids = NULL; + + /* Nothing to do if there's no parent to begin with. */ + if (!IS_OTHER_REL(prel)) + return allpartrelids; + + /* + * Traverse up to the pathrel's topmost partitioned parent, collecting + * parent relids as we go; but stop if we reach parentrel. (Normally, a + * pathrel's topmost partitioned parent is either parentrel or a UNION ALL + * appendrel child of parentrel. But when handling partitionwise joins of + * multi-level partitioning trees, we can see an append path whose + * parentrel is an intermediate partitioned table.) + */ + do + { + Relids parent_relids = NULL; + + /* + * For simple child rels, we can simply get the parent relid from + * prel->parent. But for partitionwise join and aggregate child rels, + * while we can use prel->parent to move up the tree, parent relids to + * add into 'partrelids' must be found the hard way through the + * AppendInfoInfos, because 1) a joinrel's relids may point to RTE_JOIN + * entries, 2) topmost parent grouping rel's relids field is left NULL. + */ + if (IS_SIMPLE_REL(prel)) + { + prel = prel->parent; + /* Stop once we reach the root partitioned rel. */ + if (!IS_PARTITIONED_REL(prel)) + break; + parent_relids = bms_add_members(parent_relids, prel->relids); + } + else + { + AppendRelInfo **appinfos; + int nappinfos, + i; + + appinfos = find_appinfos_by_relids(root, prel->relids, + &nappinfos); + for (i = 0; i < nappinfos; i++) + { + AppendRelInfo *appinfo = appinfos[i]; + + parent_relids = bms_add_member(parent_relids, + appinfo->parent_relid); + } + pfree(appinfos); + prel = prel->parent; + } + /* accept this level as an interesting parent */ + partrelids = bms_add_members(partrelids, parent_relids); + if (prel == parentrel) + break; /* don't traverse above parentrel */ + } while (IS_OTHER_REL(prel)); + + if (partrelids == NULL) + return allpartrelids; + + return add_part_relids(allpartrelids, partrelids); +} + +/* + * add_part_relids + * Add new info to a list of Bitmapsets of partitioned relids. + * + * Within 'allpartrelids', there is one Bitmapset for each topmost parent + * partitioned rel. Each Bitmapset contains the RT indexes of the topmost + * parent as well as its relevant non-leaf child partitions. Since (by + * construction of the rangetable list) parent partitions must have lower + * RT indexes than their children, we can distinguish the topmost parent + * as being the lowest set bit in the Bitmapset. + * + * 'partrelids' contains the RT indexes of a parent partitioned rel, and + * possibly some non-leaf children, that are newly identified as parents of + * some subpath rel passed to make_partition_pruneinfo(). These are added + * to an appropriate member of 'allpartrelids'. + * + * Note that the list contains only RT indexes of partitioned tables that + * are parents of some scan-level relation appearing in the 'subpaths' that + * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are + * not allowed to be higher than the 'parentrel' associated with the append + * path. In this way, we avoid expending cycles on partitioned rels that + * can't contribute useful pruning information for the problem at hand. + * (It is possible for 'parentrel' to be a child partitioned table, and it + * is also possible for scan-level relations to be child partitioned tables + * rather than leaf partitions. Hence we must construct this relation set + * with reference to the particular append path we're dealing with, rather + * than looking at the full partitioning structure represented in the + * RelOptInfos.) + */ +static List * +add_part_relids(List *allpartrelids, Bitmapset *partrelids) +{ + Index targetpart; + ListCell *lc; + + /* We can easily get the lowest set bit this way: */ + targetpart = bms_next_member(partrelids, -1); + Assert(targetpart > 0); + + /* Look for a matching topmost parent */ + foreach(lc, allpartrelids) + { + Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); + Index currtarget = bms_next_member(currpartrelids, -1); + + if (targetpart == currtarget) + { + /* Found a match, so add any new RT indexes to this hierarchy */ + currpartrelids = bms_add_members(currpartrelids, partrelids); + lfirst(lc) = currpartrelids; + return allpartrelids; + } + } + /* No match, so add the new partition hierarchy to the list */ + return lappend(allpartrelids, partrelids); +} diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c index 510145e3c0..3557e07082 100644 --- a/src/backend/partitioning/partprune.c +++ b/src/backend/partitioning/partprune.c @@ -138,7 +138,6 @@ typedef struct PruneStepResult } PruneStepResult; -static List *add_part_relids(List *allpartrelids, Bitmapset *partrelids); static List *make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *prunequal, @@ -221,33 +220,32 @@ static void partkey_datum_from_expr(PartitionPruneContext *context, * of scan paths for its child rels. * 'prunequal' is a list of potential pruning quals (i.e., restriction * clauses that are applicable to the appendrel). + * 'allpartrelids' contains Bitmapsets of RT indexes of partitioned parents + * whose partitions' Paths are in 'subpaths'; there's one Bitmapset for every + * partition tree involved. */ int make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, List *subpaths, - List *prunequal) + List *prunequal, + List *allpartrelids) { PartitionPruneInfo *pruneinfo; Bitmapset *allmatchedsubplans = NULL; - List *allpartrelids; List *prunerelinfos; int *relid_subplan_map; ListCell *lc; int i; + Assert(list_length(allpartrelids) > 0); + /* - * Scan the subpaths to see which ones are scans of partition child - * relations, and identify their parent partitioned rels. (Note: we must - * restrict the parent partitioned rels to be parentrel or children of - * parentrel, otherwise we couldn't translate prunequal to match.) - * - * Also construct a temporary array to map from partition-child-relation - * relid to the index in 'subpaths' of the scan plan for that partition. + * Construct a temporary array to map from partition-child-relation relid + * to the index in 'subpaths' of the scan plan for that partition. * (Use of "subplan" rather than "subpath" is a bit of a misnomer, but * we'll let it stand.) For convenience, we use 1-based indexes here, so * that zero can represent an un-filled array entry. */ - allpartrelids = NIL; relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size); i = 1; @@ -256,50 +254,9 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, Path *path = (Path *) lfirst(lc); RelOptInfo *pathrel = path->parent; - /* We don't consider partitioned joins here */ - if (pathrel->reloptkind == RELOPT_OTHER_MEMBER_REL) - { - RelOptInfo *prel = pathrel; - Bitmapset *partrelids = NULL; - - /* - * Traverse up to the pathrel's topmost partitioned parent, - * collecting parent relids as we go; but stop if we reach - * parentrel. (Normally, a pathrel's topmost partitioned parent - * is either parentrel or a UNION ALL appendrel child of - * parentrel. But when handling partitionwise joins of - * multi-level partitioning trees, we can see an append path whose - * parentrel is an intermediate partitioned table.) - */ - do - { - AppendRelInfo *appinfo; - - Assert(prel->relid < root->simple_rel_array_size); - appinfo = root->append_rel_array[prel->relid]; - prel = find_base_rel(root, appinfo->parent_relid); - if (!IS_PARTITIONED_REL(prel)) - break; /* reached a non-partitioned parent */ - /* accept this level as an interesting parent */ - partrelids = bms_add_member(partrelids, prel->relid); - if (prel == parentrel) - break; /* don't traverse above parentrel */ - } while (prel->reloptkind == RELOPT_OTHER_MEMBER_REL); - - if (partrelids) - { - /* - * Found some relevant parent partitions, which may or may not - * overlap with partition trees we already found. Add new - * information to the allpartrelids list. - */ - allpartrelids = add_part_relids(allpartrelids, partrelids); - /* Also record the subplan in relid_subplan_map[] */ - /* No duplicates please */ - Assert(relid_subplan_map[pathrel->relid] == 0); - relid_subplan_map[pathrel->relid] = i; - } - } + /* No duplicates please */ + Assert(relid_subplan_map[pathrel->relid] == 0); + relid_subplan_map[pathrel->relid] = i; i++; } @@ -368,63 +325,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel, return list_length(root->partPruneInfos) - 1; } -/* - * add_part_relids - * Add new info to a list of Bitmapsets of partitioned relids. - * - * Within 'allpartrelids', there is one Bitmapset for each topmost parent - * partitioned rel. Each Bitmapset contains the RT indexes of the topmost - * parent as well as its relevant non-leaf child partitions. Since (by - * construction of the rangetable list) parent partitions must have lower - * RT indexes than their children, we can distinguish the topmost parent - * as being the lowest set bit in the Bitmapset. - * - * 'partrelids' contains the RT indexes of a parent partitioned rel, and - * possibly some non-leaf children, that are newly identified as parents of - * some subpath rel passed to make_partition_pruneinfo(). These are added - * to an appropriate member of 'allpartrelids'. - * - * Note that the list contains only RT indexes of partitioned tables that - * are parents of some scan-level relation appearing in the 'subpaths' that - * make_partition_pruneinfo() is dealing with. Also, "topmost" parents are - * not allowed to be higher than the 'parentrel' associated with the append - * path. In this way, we avoid expending cycles on partitioned rels that - * can't contribute useful pruning information for the problem at hand. - * (It is possible for 'parentrel' to be a child partitioned table, and it - * is also possible for scan-level relations to be child partitioned tables - * rather than leaf partitions. Hence we must construct this relation set - * with reference to the particular append path we're dealing with, rather - * than looking at the full partitioning structure represented in the - * RelOptInfos.) - */ -static List * -add_part_relids(List *allpartrelids, Bitmapset *partrelids) -{ - Index targetpart; - ListCell *lc; - - /* We can easily get the lowest set bit this way: */ - targetpart = bms_next_member(partrelids, -1); - Assert(targetpart > 0); - - /* Look for a matching topmost parent */ - foreach(lc, allpartrelids) - { - Bitmapset *currpartrelids = (Bitmapset *) lfirst(lc); - Index currtarget = bms_next_member(currpartrelids, -1); - - if (targetpart == currtarget) - { - /* Found a match, so add any new RT indexes to this hierarchy */ - currpartrelids = bms_add_members(currpartrelids, partrelids); - lfirst(lc) = currpartrelids; - return allpartrelids; - } - } - /* No match, so add the new partition hierarchy to the list */ - return lappend(allpartrelids, partrelids); -} - /* * make_partitionedrel_pruneinfo * Build a List of PartitionedRelPruneInfos, one for each interesting diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 659bd05c0c..a0bb16cff4 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -270,6 +270,13 @@ typedef struct Append List *appendplans; int nasyncplans; /* # of asynchronous plans */ + /* + * RTIs of all partitioned tables whose children are scanned by + * appendplans. The list contains a bitmapset for every partition tree + * covered by this Append. + */ + List *allpartrelids; + /* * All 'appendplans' preceding this index are non-partial plans. All * 'appendplans' from this index onwards are partial plans. @@ -294,6 +301,13 @@ typedef struct MergeAppend List *mergeplans; + /* + * RTIs of all partitioned tables whose children are scanned by + * mergeplans. The list contains a bitmapset for every partition tree + * covered by this MergeAppend. + */ + List *allpartrelids; + /* these fields are just like the sort-key info in struct Sort: */ /* number of sort-key columns */ diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h index a05f91f77d..1621a7319a 100644 --- a/src/include/optimizer/appendinfo.h +++ b/src/include/optimizer/appendinfo.h @@ -46,5 +46,8 @@ extern void add_row_identity_columns(PlannerInfo *root, Index rtindex, RangeTblEntry *target_rte, Relation target_relation); extern void distribute_row_identity_vars(PlannerInfo *root); +extern List *add_append_subpath_partrelids(PlannerInfo *root, Path *subpath, + RelOptInfo *parentrel, + List *allpartrelids); #endif /* APPENDINFO_H */ diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h index c0d6889d47..2d907d31d4 100644 --- a/src/include/partitioning/partprune.h +++ b/src/include/partitioning/partprune.h @@ -73,7 +73,8 @@ typedef struct PartitionPruneContext extern int make_partition_pruneinfo(struct PlannerInfo *root, struct RelOptInfo *parentrel, List *subpaths, - List *prunequal); + List *prunequal, + List *allpartrelids); extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel); extern Bitmapset *get_matching_partitions(PartitionPruneContext *context, List *pruning_steps); -- 2.35.3 [application/octet-stream] v38-0003-Track-opened-range-table-relations-in-a-List-in-.patch (2.3K, 3-v38-0003-Track-opened-range-table-relations-in-a-List-in-.patch) download | inline diff: From 7859c3ee10dbe81606241478ef085aeb7d45a95d Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Mon, 13 Mar 2023 15:59:38 +0900 Subject: [PATCH v38 3/3] Track opened range table relations in a List in EState This makes ExecCloseRangeTableRelations faster when there are many relations in the range table but only a few are opened during execution, such as when run-time pruning kicks in on an Append containing 1000s of partition subplans. --- src/backend/executor/execMain.c | 9 +++++---- src/backend/executor/execUtils.c | 2 ++ src/include/nodes/execnodes.h | 2 ++ 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 0366be9fd6..94f8324cff 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -1630,12 +1630,13 @@ ExecCloseResultRelations(EState *estate) void ExecCloseRangeTableRelations(EState *estate) { - int i; + ListCell *lc; - for (i = 0; i < estate->es_range_table_size; i++) + foreach(lc, estate->es_opened_relations) { - if (estate->es_relations[i]) - table_close(estate->es_relations[i], NoLock); + Relation rel = lfirst(lc); + + table_close(rel, NoLock); } } diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index a485e7dfc5..f7053072d9 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -829,6 +829,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) } estate->es_relations[rti - 1] = rel; + estate->es_opened_relations = lappend(estate->es_opened_relations, + rel); } return rel; diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index dfa72848c7..984fd2e423 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -619,6 +619,8 @@ typedef struct EState Index es_range_table_size; /* size of the range table arrays */ Relation *es_relations; /* Array of per-range-table-entry Relation * pointers, or NULL if not yet opened */ + List *es_opened_relations; /* List of non-NULL entries in + * es_relations in no specific order */ struct ExecRowMark **es_rowmarks; /* Array of per-range-table-entry * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ -- 2.35.3 [application/octet-stream] v38-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch (134.5K, 4-v38-0002-Move-AcquireExecutorLocks-s-responsibility-into-.patch) download | inline diff: From 3476743ef207ba23f0c366aba509c439a4cdf559 Mon Sep 17 00:00:00 2001 From: amitlan <[email protected]> Date: Fri, 20 Jan 2023 16:52:31 +0900 Subject: [PATCH v38 2/3] Move AcquireExecutorLocks()'s responsibility into the executor This commit introduces a new executor flag EXEC_FLAG_GET_LOCKS that should be passed in eflags to ExecutorStart() if the PlannedStmt comes from a CachedPlan. When set, the executor will take locks on any relations referenced in the plan nodes that need to be initialized for execution. That excludes any partitions that can be pruned during the executor initialization phase, that is, based on the values of only the external (PARAM_EXTERN) parameters. Relations that are not explicitly mentioned in the plan tree, such as views and non-leaf partition parents whose children are mentioned in Append/MergeAppend nodes, are locked separately. After taking each lock, the executor calls CachedPlanStillValid() to check if CachedPlan.is_valid has been reset by PlanCacheRelCallback() due to concurrent modification of relations referenced in the plan. If it is found that the CachedPlan is indeed invalid, the recursive ExecInitNode() traversal is aborted at that point. To allow the proper cleanup of such a partially initialized planstate tree, ExecEndNode() subroutines of various plan nodes have been fixed to account for potentially uninitialized fields. It is the caller's (of ExecutorStart()) responsibility to call ExecutorEnd() even on a QueryDesc containing such a partially initialized PlanState tree. Call sites that use plancache (GetCachedPlan) to get the plan trees to pass to the executor for execution should now be prepared to handle the case that the plan tree may be flagged by the executor as stale as described above. To that end, this commit refactors the relevant code sites to move the ExecutorStart() call closer to the GetCachedPlan() call to reduce the friction in the cases where replanning is needed due to a CachedPlan being marked stale in this manner. Callers must check that QueryDesc.plan_valid is true before passing it on to ExecutorRun() for execution. PortalStart() now performs CreateQueryDesc() and ExecutorStart() for all portal strategies, including those pertaining to multiple queries. The QueryDescs for strategies handled by PortalRunMulti() are remembered in the Portal in a new List field 'qdescs', allocated in a new memory context 'queryContext'. This new arrangment is to make it easier to discard and recreate a Portal if the CachedPlan goes stale during setup. --- contrib/postgres_fdw/postgres_fdw.c | 4 + doc/src/sgml/custom-scan.sgml | 4 +- doc/src/sgml/fdwhandler.sgml | 4 +- src/backend/commands/copyto.c | 4 +- src/backend/commands/createas.c | 2 +- src/backend/commands/explain.c | 150 ++++++--- src/backend/commands/extension.c | 2 + src/backend/commands/matview.c | 3 +- src/backend/commands/portalcmds.c | 16 +- src/backend/commands/prepare.c | 32 +- src/backend/executor/execMain.c | 89 ++++- src/backend/executor/execParallel.c | 8 +- src/backend/executor/execPartition.c | 15 + src/backend/executor/execProcnode.c | 16 + src/backend/executor/execUtils.c | 60 +++- src/backend/executor/functions.c | 2 + src/backend/executor/nodeAgg.c | 23 +- src/backend/executor/nodeAppend.c | 23 +- src/backend/executor/nodeBitmapAnd.c | 10 +- src/backend/executor/nodeBitmapHeapscan.c | 10 +- src/backend/executor/nodeBitmapIndexscan.c | 2 + src/backend/executor/nodeBitmapOr.c | 10 +- src/backend/executor/nodeCtescan.c | 6 +- src/backend/executor/nodeCustom.c | 13 +- src/backend/executor/nodeForeignscan.c | 28 +- src/backend/executor/nodeFunctionscan.c | 3 +- src/backend/executor/nodeGather.c | 2 + src/backend/executor/nodeGatherMerge.c | 2 + src/backend/executor/nodeGroup.c | 5 +- src/backend/executor/nodeHash.c | 2 + src/backend/executor/nodeHashjoin.c | 13 +- src/backend/executor/nodeIncrementalSort.c | 14 +- src/backend/executor/nodeIndexonlyscan.c | 7 +- src/backend/executor/nodeIndexscan.c | 7 +- src/backend/executor/nodeLimit.c | 2 + src/backend/executor/nodeLockRows.c | 2 + src/backend/executor/nodeMaterial.c | 5 +- src/backend/executor/nodeMemoize.c | 12 +- src/backend/executor/nodeMergeAppend.c | 23 +- src/backend/executor/nodeMergejoin.c | 10 +- src/backend/executor/nodeModifyTable.c | 13 +- .../executor/nodeNamedtuplestorescan.c | 3 +- src/backend/executor/nodeNestloop.c | 7 +- src/backend/executor/nodeProjectSet.c | 5 +- src/backend/executor/nodeRecursiveunion.c | 4 + src/backend/executor/nodeResult.c | 5 +- src/backend/executor/nodeSamplescan.c | 5 +- src/backend/executor/nodeSeqscan.c | 5 +- src/backend/executor/nodeSetOp.c | 5 +- src/backend/executor/nodeSort.c | 8 +- src/backend/executor/nodeSubqueryscan.c | 5 +- src/backend/executor/nodeTableFuncscan.c | 3 +- src/backend/executor/nodeTidrangescan.c | 5 +- src/backend/executor/nodeTidscan.c | 5 +- src/backend/executor/nodeUnique.c | 5 +- src/backend/executor/nodeValuesscan.c | 3 +- src/backend/executor/nodeWindowAgg.c | 55 +++- src/backend/executor/nodeWorktablescan.c | 3 +- src/backend/executor/spi.c | 53 ++- src/backend/nodes/outfuncs.c | 1 + src/backend/nodes/readfuncs.c | 1 + src/backend/optimizer/plan/planner.c | 1 + src/backend/optimizer/plan/setrefs.c | 5 + src/backend/rewrite/rewriteHandler.c | 7 +- src/backend/storage/lmgr/lmgr.c | 45 +++ src/backend/tcop/postgres.c | 13 +- src/backend/tcop/pquery.c | 311 ++++++++++-------- src/backend/utils/cache/lsyscache.c | 21 ++ src/backend/utils/cache/plancache.c | 134 ++------ src/backend/utils/mmgr/portalmem.c | 6 + src/include/commands/explain.h | 7 +- src/include/executor/execdesc.h | 5 + src/include/executor/executor.h | 16 + src/include/nodes/execnodes.h | 2 + src/include/nodes/pathnodes.h | 3 + src/include/nodes/plannodes.h | 3 + src/include/storage/lmgr.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/plancache.h | 14 + src/include/utils/portal.h | 4 + src/test/modules/delay_execution/Makefile | 3 +- .../modules/delay_execution/delay_execution.c | 63 +++- .../expected/cached-plan-replan.out | 117 +++++++ .../specs/cached-plan-replan.spec | 50 +++ 84 files changed, 1250 insertions(+), 426 deletions(-) create mode 100644 src/test/modules/delay_execution/expected/cached-plan-replan.out create mode 100644 src/test/modules/delay_execution/specs/cached-plan-replan.spec diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index f5926ab89d..93f3f8b5d1 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @@ -2659,7 +2659,11 @@ postgresBeginDirectModify(ForeignScanState *node, int eflags) /* Get info about foreign table. */ rtindex = node->resultRelInfo->ri_RangeTableIndex; if (fsplan->scan.scanrelid == 0) + { dmstate->rel = ExecOpenScanRelation(estate, rtindex, eflags); + if (!ExecPlanStillValid(estate)) + return; + } else dmstate->rel = node->ss.ss_currentRelation; table = GetForeignTable(RelationGetRelid(dmstate->rel)); diff --git a/doc/src/sgml/custom-scan.sgml b/doc/src/sgml/custom-scan.sgml index 93d96f2f56..c755e2d681 100644 --- a/doc/src/sgml/custom-scan.sgml +++ b/doc/src/sgml/custom-scan.sgml @@ -275,7 +275,9 @@ void (*EndCustomScan) (CustomScanState *node); </programlisting> Clean up any private data associated with the <literal>CustomScanState</literal>. This method is required, but it does not need to do anything if there is - no associated data or it will be cleaned up automatically. + no associated data or it will be cleaned up automatically. Note that this + may be called even if the corresponding <function>BeginCustomScan</function> + was not called by <function>ExecInitCustomScan</function>. </para> <para> diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index ac1717bc3c..a97dcd9054 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml/fdwhandler.sgml @@ -289,7 +289,9 @@ EndForeignScan(ForeignScanState *node); End the scan and release resources. It is normally not important to release palloc'd memory, but for example open files and connections - to remote servers should be cleaned up. + to remote servers should be cleaned up. Note that this may be called + even if the corresponding <function>BeginForeignScan</function> was + not called by <function>ExecInitForeignScan</function>. </para> </sect2> diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c index beea1ac687..e9f77d5711 100644 --- a/src/backend/commands/copyto.c +++ b/src/backend/commands/copyto.c @@ -558,7 +558,8 @@ BeginCopyTo(ParseState *pstate, ((DR_copy *) dest)->cstate = cstate; /* Create a QueryDesc requesting no output */ - cstate->queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + cstate->queryDesc = CreateQueryDesc(plan, NULL, + pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); @@ -569,6 +570,7 @@ BeginCopyTo(ParseState *pstate, * ExecutorStart computes a result tupdesc for us */ ExecutorStart(cstate->queryDesc, 0); + Assert(cstate->queryDesc->plan_valid); tupDesc = cstate->queryDesc->tupDesc; } diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index d6c6d514f3..a55b851574 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -325,7 +325,7 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, pstate->p_sourcetext, + queryDesc = CreateQueryDesc(plan, NULL, pstate->p_sourcetext, GetActiveSnapshot(), InvalidSnapshot, dest, params, queryEnv, 0); diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index 878d2fd172..826a47af0a 100644 --- a/src/backend/commands/explain.c +++ b/src/backend/commands/explain.c @@ -393,6 +393,7 @@ ExplainOneQuery(Query *query, int cursorOptions, else { PlannedStmt *plan; + QueryDesc *queryDesc; instr_time planstart, planduration; BufferUsage bufusage_start, @@ -415,12 +416,95 @@ ExplainOneQuery(Query *query, int cursorOptions, BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start); } + queryDesc = ExplainQueryDesc(plan, NULL, queryString, into, es, + params, queryEnv); + Assert(queryDesc); + /* run it (if needed) and produce output */ - ExplainOnePlan(plan, into, es, queryString, params, queryEnv, + ExplainOnePlan(queryDesc, into, es, queryString, params, queryEnv, &planduration, (es->buffers ? &bufusage : NULL)); } } +/* + * ExplainQueryDesc + * Set up QueryDesc for EXPLAINing a given plan + * + * This returns NULL if cplan is found to have been invalidated since its + * creation. + */ +QueryDesc * +ExplainQueryDesc(PlannedStmt *stmt, CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv) +{ + QueryDesc *queryDesc; + DestReceiver *dest; + int eflags; + int instrument_option = 0; + + /* + * Normally we discard the query's output, but if explaining CREATE TABLE + * AS, we'd better use the appropriate tuple receiver. + */ + if (into) + dest = CreateIntoRelDestReceiver(into); + else + dest = None_Receiver; + + if (es->analyze && es->timing) + instrument_option |= INSTRUMENT_TIMER; + else if (es->analyze) + instrument_option |= INSTRUMENT_ROWS; + + if (es->buffers) + instrument_option |= INSTRUMENT_BUFFERS; + if (es->wal) + instrument_option |= INSTRUMENT_WAL; + + /* + * Use a snapshot with an updated command ID to ensure this query sees + * results of any previously executed queries. + */ + PushCopiedSnapshot(GetActiveSnapshot()); + UpdateActiveSnapshotCommandId(); + + /* Create a QueryDesc for the query */ + queryDesc = CreateQueryDesc(stmt, cplan, queryString, + GetActiveSnapshot(), InvalidSnapshot, + dest, params, queryEnv, instrument_option); + + /* Select execution options */ + if (es->analyze) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_EXPLAIN_ONLY; + if (es->generic) + eflags |= EXEC_FLAG_EXPLAIN_GENERIC; + if (into) + eflags |= GetIntoRelEFlags(into); + + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + /* + * Call ExecutorStart to prepare the plan for execution. A cached plan + * may get invalidated as we're doing that. + */ + ExecutorStart(queryDesc, eflags); + if (!queryDesc->plan_valid) + { + /* Clean up. */ + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); + PopActiveSnapshot(); + return NULL; + } + + return queryDesc; +} + /* * ExplainOneUtility - * print out the execution plan for one utility statement @@ -524,29 +608,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, * to call it. */ void -ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, +ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, const BufferUsage *bufusage) { - DestReceiver *dest; - QueryDesc *queryDesc; instr_time starttime; double totaltime = 0; - int eflags; - int instrument_option = 0; - Assert(plannedstmt->commandType != CMD_UTILITY); - - if (es->analyze && es->timing) - instrument_option |= INSTRUMENT_TIMER; - else if (es->analyze) - instrument_option |= INSTRUMENT_ROWS; - - if (es->buffers) - instrument_option |= INSTRUMENT_BUFFERS; - if (es->wal) - instrument_option |= INSTRUMENT_WAL; + Assert(queryDesc->plannedstmt->commandType != CMD_UTILITY); /* * We always collect timing for the entire statement, even when node-level @@ -555,40 +626,6 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es, */ INSTR_TIME_SET_CURRENT(starttime); - /* - * Use a snapshot with an updated command ID to ensure this query sees - * results of any previously executed queries. - */ - PushCopiedSnapshot(GetActiveSnapshot()); - UpdateActiveSnapshotCommandId(); - - /* - * Normally we discard the query's output, but if explaining CREATE TABLE - * AS, we'd better use the appropriate tuple receiver. - */ - if (into) - dest = CreateIntoRelDestReceiver(into); - else - dest = None_Receiver; - - /* Create a QueryDesc for the query */ - queryDesc = CreateQueryDesc(plannedstmt, queryString, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, instrument_option); - - /* Select execution options */ - if (es->analyze) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_EXPLAIN_ONLY; - if (es->generic) - eflags |= EXEC_FLAG_EXPLAIN_GENERIC; - if (into) - eflags |= GetIntoRelEFlags(into); - - /* call ExecutorStart to prepare the plan for execution */ - ExecutorStart(queryDesc, eflags); - /* Execute the plan for statistics if asked for */ if (es->analyze) { @@ -4862,6 +4899,17 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es) } } +/* + * Discard output buffer for a fresh restart. + */ +void +ExplainResetOutput(ExplainState *es) +{ + Assert(es->str); + resetStringInfo(es->str); + ExplainBeginOutput(es); +} + /* * Emit the start-of-output boilerplate. * diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index 0eabe18335..5a76343123 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -797,11 +797,13 @@ execute_sql_string(const char *sql) QueryDesc *qdesc; qdesc = CreateQueryDesc(stmt, + NULL, sql, GetActiveSnapshot(), NULL, dest, NULL, NULL, 0); ExecutorStart(qdesc, 0); + Assert(qdesc->plan_valid); ExecutorRun(qdesc, ForwardScanDirection, 0, true); ExecutorFinish(qdesc); ExecutorEnd(qdesc); diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c index c00b9df3e3..80f2c38b35 100644 --- a/src/backend/commands/matview.c +++ b/src/backend/commands/matview.c @@ -409,12 +409,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query, UpdateActiveSnapshotCommandId(); /* Create a QueryDesc, redirecting output to our tuple receiver */ - queryDesc = CreateQueryDesc(plan, queryString, + queryDesc = CreateQueryDesc(plan, NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, dest, NULL, NULL, 0); /* call ExecutorStart to prepare the plan for execution */ ExecutorStart(queryDesc, 0); + Assert(queryDesc->plan_valid); /* run the plan */ ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 8a3cf98cce..3c34ab4351 100644 --- a/src/backend/commands/portalcmds.c +++ b/src/backend/commands/portalcmds.c @@ -146,6 +146,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa PortalStart(portal, params, 0, GetActiveSnapshot()); Assert(portal->strategy == PORTAL_ONE_SELECT); + Assert(portal->plan_valid); /* * We're done; the query won't actually be run until PerformPortalFetch is @@ -249,6 +250,17 @@ PerformPortalClose(const char *name) PortalDrop(portal, false); } +/* + * Release a portal's QueryDesc. + */ +void +PortalQueryFinish(QueryDesc *queryDesc) +{ + ExecutorFinish(queryDesc); + ExecutorEnd(queryDesc); + FreeQueryDesc(queryDesc); +} + /* * PortalCleanup * @@ -295,9 +307,7 @@ PortalCleanup(Portal portal) if (portal->resowner) CurrentResourceOwner = portal->resowner; - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - FreeQueryDesc(queryDesc); + PortalQueryFinish(queryDesc); CurrentResourceOwner = saveResourceOwner; } diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index 18f70319fc..c9070ed97f 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -183,6 +183,7 @@ ExecuteQuery(ParseState *pstate, paramLI = EvaluateParams(pstate, entry, stmt->params, estate); } +replan: /* Create a new portal to run the query in */ portal = CreateNewPortal(); /* Don't display the portal in pg_cursors, it is for internal use only */ @@ -251,10 +252,19 @@ ExecuteQuery(ParseState *pstate, } /* - * Run the portal as appropriate. + * Run the portal as appropriate. If the portal contains a cached plan, it + * must be recreated if portal->plan_valid is false which tells that the + * cached plan was found to have been invalidated when initializing one of + * the plan trees contained in it. */ PortalStart(portal, paramLI, eflags, GetActiveSnapshot()); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + (void) PortalRun(portal, count, false, true, dest, dest, qc); PortalDrop(portal, false); @@ -574,7 +584,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, { PreparedStatement *entry; const char *query_string; - CachedPlan *cplan; + CachedPlan *cplan = NULL; List *plan_list; ListCell *p; ParamListInfo paramLI = NULL; @@ -618,6 +628,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, } /* Replan if needed, and acquire a transient refcount */ +replan: cplan = GetCachedPlan(entry->plansource, paramLI, CurrentResourceOwner, queryEnv); @@ -639,8 +650,21 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es, PlannedStmt *pstmt = lfirst_node(PlannedStmt, p); if (pstmt->commandType != CMD_UTILITY) - ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv, - &planduration, (es->buffers ? &bufusage : NULL)); + { + QueryDesc *queryDesc; + + queryDesc = ExplainQueryDesc(pstmt, cplan, queryString, + into, es, paramLI, queryEnv); + if (queryDesc == NULL) + { + ExplainResetOutput(es); + ReleaseCachedPlan(cplan, CurrentResourceOwner); + goto replan; + } + ExplainOnePlan(queryDesc, into, es, query_string, paramLI, + queryEnv, &planduration, + (es->buffers ? &bufusage : NULL)); + } else ExplainOneUtility(pstmt->utilityStmt, into, es, query_string, paramLI, queryEnv); diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 1b007dc32c..0366be9fd6 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -126,11 +126,32 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree); * get control when ExecutorStart is called. Such a plugin would * normally call standard_ExecutorStart(). * + * Normally, the plan tree given in queryDesc->plannedstmt is known to be + * valid in that *all* relations contained in plannedstmt->relationOids have + * already been locked. That may not be the case however if the plannedstmt + * comes from a CachedPlan, one given in queryDesc->cplan, in which case only + * some of the relations referenced in the plan would have been locked; to + * wit, those that AcquirePlannerLocks() deems necessary. Locks necessary + * to fully validate such a plan tree, including relations that are added by + * the planner, will be taken when initializing the plan tree in InitPlan(); + * the the caller must have set the EXEC_FLAG_GET_LOCKS bit in eflags. If the + * CachedPlan gets invalidated as these locks are taken, plan tree + * initialization is suspended at the point when such invalidation is first + * detected and InitPlan() returns after setting queryDesc->plan_valid to + * false. queryDesc->planstate would be pointing to a potentially partially + * initialized PlanState tree in that case. Callers must retry the execution + * with a freshly created CachedPlan in that case, after properly freeing the + * partially valid QueryDesc. * ---------------------------------------------------------------- */ void ExecutorStart(QueryDesc *queryDesc, int eflags) { + /* Take locks if the plan tree comes from a CachedPlan. */ + Assert(queryDesc->cplan == NULL || + (CachedPlanStillValid(queryDesc->cplan) && + (eflags & EXEC_FLAG_GET_LOCKS) != 0)); + /* * In some cases (e.g. an EXECUTE statement) a query execution will skip * parse analysis, which means that the query_id won't be reported. Note @@ -582,6 +603,16 @@ ExecCheckPermissions(List *rangeTable, List *rteperminfos, RTEPermissionInfo *perminfo = lfirst_node(RTEPermissionInfo, l); Assert(OidIsValid(perminfo->relid)); + + /* + * Relations whose permissions need to be checked must already have + * been locked by the parser or by AcquirePlannerLocks() if a + * cached plan is being executed. + * XXX shouldn't we skip calling ExecCheckPermissions from InitPlan + * in a parallel worker? + */ + Assert(CheckRelLockedByMe(perminfo->relid, AccessShareLock, true) || + IsParallelWorker()); result = ExecCheckOneRelPerms(perminfo); if (!result) { @@ -785,12 +816,19 @@ ExecCheckXactReadOnly(PlannedStmt *plannedstmt) PreventCommandIfParallelMode(CreateCommandName((Node *) plannedstmt)); } - /* ---------------------------------------------------------------- * InitPlan * * Initializes the query plan: open files, allocate storage * and start up the rule manager + * + * If queryDesc contains a CachedPlan, this takes locks on relations. + * If any of those relations have undergone concurrent schema changes + * between successfully performing RevalidateCachedQuery() on the + * containing CachedPlanSource and here, locking those relations would + * invalidate the CachedPlan by way of PlanCacheRelCallback(). In that + * case, queryDesc->plan_valid would be set to false to tell the caller + * to retry after creating a new CachedPlan. * ---------------------------------------------------------------- */ static void @@ -801,20 +839,32 @@ InitPlan(QueryDesc *queryDesc, int eflags) Plan *plan = plannedstmt->planTree; List *rangeTable = plannedstmt->rtable; EState *estate = queryDesc->estate; - PlanState *planstate; + PlanState *planstate = NULL; TupleDesc tupType; ListCell *l; int i; /* - * Do permissions checks + * Set up range table in EState. */ - ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); + ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + + /* Make sure ExecPlanStillValid() can work. */ + estate->es_cachedplan = queryDesc->cplan; /* - * initialize the node's execution state + * Lock any views that were mentioned in the query if needed. View + * relations must be locked separately like this, because they are not + * referenced in the plan tree. */ - ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos); + ExecLockViewRelations(plannedstmt->viewRelations, estate); + if (!ExecPlanStillValid(estate)) + goto failed; + + /* + * Do permissions checks + */ + ExecCheckPermissions(rangeTable, plannedstmt->permInfos, true); estate->es_plannedstmt = plannedstmt; estate->es_part_prune_infos = plannedstmt->partPruneInfos; @@ -849,6 +899,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) case ROW_MARK_KEYSHARE: case ROW_MARK_REFERENCE: relation = ExecGetRangeTableRelation(estate, rc->rti); + if (!ExecPlanStillValid(estate)) + goto failed; break; case ROW_MARK_COPY: /* no physical table access is required */ @@ -919,6 +971,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) estate->es_subplanstates = lappend(estate->es_subplanstates, subplanstate); + if (!ExecPlanStillValid(estate)) + goto failed; i++; } @@ -929,6 +983,8 @@ InitPlan(QueryDesc *queryDesc, int eflags) * processing tuples. */ planstate = ExecInitNode(plan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto failed; /* * Get the tuple descriptor describing the type of tuples to return. @@ -972,6 +1028,17 @@ InitPlan(QueryDesc *queryDesc, int eflags) queryDesc->tupDesc = tupType; queryDesc->planstate = planstate; + queryDesc->plan_valid = true; + return; + +failed: + /* + * Plan initialization failed. Mark QueryDesc as such. Note that we do + * set planstate, even if it may only be partially initialized, so that + * ExecEndPlan() can process it. + */ + queryDesc->planstate = planstate; + queryDesc->plan_valid = false; } /* @@ -1389,7 +1456,7 @@ ExecGetAncestorResultRels(EState *estate, ResultRelInfo *resultRelInfo) /* * All ancestors up to the root target relation must have been - * locked by the planner or AcquireExecutorLocks(). + * locked. */ ancRel = table_open(ancOid, NoLock); rInfo = makeNode(ResultRelInfo); @@ -2797,7 +2864,8 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) * Child EPQ EStates share the parent's copy of unchanging state such as * the snapshot, rangetable, and external Param info. They need their own * copies of local state, including a tuple table, es_param_exec_vals, - * result-rel info, etc. + * result-rel info, etc. Also, we don't pass the parent't copy of the + * CachedPlan, because no new locks will be taken for EvalPlanQual(). */ rcestate->es_direction = ForwardScanDirection; rcestate->es_snapshot = parentestate->es_snapshot; @@ -2884,6 +2952,7 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree) PlanState *subplanstate; subplanstate = ExecInitNode(subplan, rcestate, 0); + Assert(ExecPlanStillValid(rcestate)); rcestate->es_subplanstates = lappend(rcestate->es_subplanstates, subplanstate); } @@ -2937,6 +3006,10 @@ EvalPlanQualEnd(EPQState *epqstate) MemoryContext oldcontext; ListCell *l; + /* Nothing to do if EvalPlanQualInit() wasn't done to begin with. */ + if (epqstate->parentestate == NULL) + return; + rtsize = epqstate->parentestate->es_range_table_size; /* diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c index aa3f283453..df4cc5ddaf 100644 --- a/src/backend/executor/execParallel.c +++ b/src/backend/executor/execParallel.c @@ -1249,8 +1249,13 @@ ExecParallelGetQueryDesc(shm_toc *toc, DestReceiver *receiver, paramspace = shm_toc_lookup(toc, PARALLEL_KEY_PARAMLISTINFO, false); paramLI = RestoreParamList(¶mspace); - /* Create a QueryDesc for the query. */ + /* + * Create a QueryDesc for the query. Note that no CachedPlan is available + * here even if the containing plan tree may have come from one in the + * leader. + */ return CreateQueryDesc(pstmt, + NULL, queryString, GetActiveSnapshot(), InvalidSnapshot, receiver, paramLI, NULL, instrument_options); @@ -1432,6 +1437,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc) /* Start up the executor */ queryDesc->plannedstmt->jitFlags = fpes->jit_flags; ExecutorStart(queryDesc, fpes->eflags); + Assert(queryDesc->plan_valid); /* Special executor initialization steps for parallel workers */ queryDesc->planstate->state->es_query_dsa = area; diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 9799968a42..3425ffcca7 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -513,6 +513,12 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate, oldcxt = MemoryContextSwitchTo(proute->memcxt); + /* + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. + */ partrel = table_open(partOid, RowExclusiveLock); leaf_part_rri = makeNode(ResultRelInfo); @@ -1111,6 +1117,11 @@ ExecInitPartitionDispatchInfo(EState *estate, * Only sub-partitioned tables need to be locked here. The root * partitioned table will already have been locked as it's referenced in * the query's rtable. + * + * Note that while we must check ExecPlanStillValid() for other locks taken + * during execution initialization, it is OK to not do so for partitions + * opened like this, for tuple routing, because it can't possibly + * invalidate the plan. */ if (partoid != RelationGetRelid(proute->partition_root)) rel = table_open(partoid, RowExclusiveLock); @@ -1817,6 +1828,8 @@ ExecInitPartitionPruning(PlanState *planstate, /* Create the working data structure for pruning */ prunestate = CreatePartitionPruneState(planstate, pruneinfo); + if (!ExecPlanStillValid(estate)) + return NULL; /* * Perform an initial partition prune pass, if required. @@ -1943,6 +1956,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo) * duration of this executor run. */ partrel = ExecGetRangeTableRelation(estate, pinfo->rtindex); + if (!ExecPlanStillValid(estate)) + return NULL; partkey = RelationGetPartitionKey(partrel); partdesc = PartitionDirectoryLookup(estate->es_partition_directory, partrel); diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c index 4d288bc8d4..a9b22e0f16 100644 --- a/src/backend/executor/execProcnode.c +++ b/src/backend/executor/execProcnode.c @@ -388,6 +388,9 @@ ExecInitNode(Plan *node, EState *estate, int eflags) break; } + if (!ExecPlanStillValid(estate)) + return result; + ExecSetExecProcNode(result, result->ExecProcNode); /* @@ -403,6 +406,12 @@ ExecInitNode(Plan *node, EState *estate, int eflags) Assert(IsA(subplan, SubPlan)); sstate = ExecInitSubPlan(subplan, result); subps = lappend(subps, sstate); + if (!ExecPlanStillValid(estate)) + { + /* Don't lose track of those initialized. */ + result->initPlan = subps; + return result; + } } result->initPlan = subps; @@ -551,6 +560,13 @@ MultiExecProcNode(PlanState *node) * After this operation, the query plan will not be able to be * processed any further. This should be called only after * the query plan has been fully executed. + * + * Note: Subroutines for various node types should be prepared to handle + * the cases where the corresponding ExecInitNode() subroutines may return + * early if the lock taken on relation handled by a given node caused the + * plan to be invalidated (ExecPlanStillValid() stops returnins true), in + * which case, not all of the node's PlanState struct's fields would have + * been initialized. * ---------------------------------------------------------------- */ void diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 012dbb6965..a485e7dfc5 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -804,7 +804,8 @@ ExecGetRangeTableRelation(EState *estate, Index rti) Assert(rte->rtekind == RTE_RELATION); - if (!IsParallelWorker()) + if (!IsParallelWorker() && + (estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) { /* * In a normal query, we should already have the appropriate lock, @@ -833,6 +834,61 @@ ExecGetRangeTableRelation(EState *estate, Index rti) return rel; } +/* + * ExecLockViewRelations + * Lock view relations, if any, in a given query + */ +void +ExecLockViewRelations(List *viewRelations, EState *estate) +{ + ListCell *lc; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(lc, viewRelations) + { + Index rti = lfirst_int(lc); + RangeTblEntry *rte = exec_rt_fetch(rti, estate); + + Assert(OidIsValid(rte->relid)); + Assert(rte->relkind == RELKIND_VIEW); + Assert(rte->rellockmode != NoLock); + + LockRelationOid(rte->relid, rte->rellockmode); + } +} + +/* + * ExecLockAppendNonLeafRelations + * Lock non-leaf relations whose children are scanned by a given + * Append/MergeAppend node + */ +void +ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids) +{ + ListCell *l; + + /* Nothing to do if no locks need to be taken. */ + if ((estate->es_top_eflags & EXEC_FLAG_GET_LOCKS) == 0) + return; + + foreach(l, allpartrelids) + { + Bitmapset *partrelids = lfirst_node(Bitmapset, l); + int i; + + i = -1; + while ((i = bms_next_member(partrelids, i)) > 0) + { + RangeTblEntry *rte = exec_rt_fetch(i, estate); + + LockRelationOid(rte->relid, rte->rellockmode); + } + } +} + /* * ExecInitResultRelation * Open relation given by the passed-in RT index and fill its @@ -848,6 +904,8 @@ ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Relation resultRelationDesc; resultRelationDesc = ExecGetRangeTableRelation(estate, rti); + if (!ExecPlanStillValid(estate)) + return; InitResultRelInfo(resultRelInfo, resultRelationDesc, rti, diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c index f55424eb5a..c88f72bc4e 100644 --- a/src/backend/executor/functions.c +++ b/src/backend/executor/functions.c @@ -838,6 +838,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) dest = None_Receiver; es->qd = CreateQueryDesc(es->stmt, + NULL, /* fmgr_sql() doesn't use CachedPlans */ fcache->src, GetActiveSnapshot(), InvalidSnapshot, @@ -863,6 +864,7 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache) else eflags = 0; /* default run-to-completion flags */ ExecutorStart(es->qd, eflags); + Assert(es->qd->plan_valid); } es->status = F_EXEC_RUN; diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 19342a420c..06e0d7d149 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -3134,15 +3134,18 @@ hashagg_reset_spill_state(AggState *aggstate) { HashAggSpill *spill = &aggstate->hash_spills[setno]; - pfree(spill->ntuples); - pfree(spill->partitions); + if (spill->ntuples) + pfree(spill->ntuples); + if (spill->partitions) + pfree(spill->partitions); } pfree(aggstate->hash_spills); aggstate->hash_spills = NULL; } /* free batches */ - list_free_deep(aggstate->hash_batches); + if (aggstate->hash_batches) + list_free_deep(aggstate->hash_batches); aggstate->hash_batches = NIL; /* close tape set */ @@ -3296,6 +3299,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags) eflags &= ~EXEC_FLAG_REWIND; outerPlan = outerPlan(node); outerPlanState(aggstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return aggstate; /* * initialize source tuple type. @@ -4336,10 +4341,13 @@ ExecEndAgg(AggState *node) { AggStatePerTrans pertrans = &node->pertrans[transno]; - for (setno = 0; setno < numGroupingSets; setno++) + if (pertrans) { - if (pertrans->sortstates[setno]) - tuplesort_end(pertrans->sortstates[setno]); + for (setno = 0; setno < numGroupingSets; setno++) + { + if (pertrans->sortstates[setno]) + tuplesort_end(pertrans->sortstates[setno]); + } } } @@ -4357,7 +4365,8 @@ ExecEndAgg(AggState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c index c185b11c67..091f979c46 100644 --- a/src/backend/executor/nodeAppend.c +++ b/src/backend/executor/nodeAppend.c @@ -109,10 +109,11 @@ AppendState * ExecInitAppend(Append *node, EState *estate, int eflags) { AppendState *appendstate = makeNode(AppendState); - PlanState **appendplanstates; + PlanState **appendplanstates = NULL; Bitmapset *validsubplans; Bitmapset *asyncplans; int nplans; + int ninited = 0; int nasyncplans; int firstvalid; int i, @@ -133,6 +134,15 @@ ExecInitAppend(Append *node, EState *estate, int eflags) appendstate->as_syncdone = false; appendstate->as_begun = false; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -148,6 +158,8 @@ ExecInitAppend(Append *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + goto early_exit; appendstate->as_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -222,11 +234,12 @@ ExecInitAppend(Append *node, EState *estate, int eflags) firstvalid = j; appendplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } appendstate->as_first_partial_plan = firstvalid; - appendstate->appendplans = appendplanstates; - appendstate->as_nplans = nplans; /* Initialize async state */ appendstate->as_asyncplans = asyncplans; @@ -276,6 +289,10 @@ ExecInitAppend(Append *node, EState *estate, int eflags) /* For parallel query, this will be overridden later. */ appendstate->choose_next_subplan = choose_next_subplan_locally; +early_exit: + appendstate->appendplans = appendplanstates; + appendstate->as_nplans = ninited; + return appendstate; } diff --git a/src/backend/executor/nodeBitmapAnd.c b/src/backend/executor/nodeBitmapAnd.c index 4c5eb2b23b..acc6c50e20 100644 --- a/src/backend/executor/nodeBitmapAnd.c +++ b/src/backend/executor/nodeBitmapAnd.c @@ -57,6 +57,7 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) BitmapAndState *bitmapandstate = makeNode(BitmapAndState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -77,8 +78,6 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) bitmapandstate->ps.plan = (Plan *) node; bitmapandstate->ps.state = estate; bitmapandstate->ps.ExecProcNode = ExecBitmapAnd; - bitmapandstate->bitmapplans = bitmapplanstates; - bitmapandstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -89,6 +88,9 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -99,6 +101,10 @@ ExecInitBitmapAnd(BitmapAnd *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmapandstate->bitmapplans = bitmapplanstates; + bitmapandstate->nplans = ninited; + return bitmapandstate; } diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c index f35df0b8bf..e6a689eefb 100644 --- a/src/backend/executor/nodeBitmapHeapscan.c +++ b/src/backend/executor/nodeBitmapHeapscan.c @@ -665,7 +665,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subplans @@ -693,7 +694,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node) /* * close heap scan */ - table_endscan(scanDesc); + if (scanDesc) + table_endscan(scanDesc); } /* ---------------------------------------------------------------- @@ -763,11 +765,15 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * initialize child nodes */ outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * get the scan type from the relation descriptor. diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c index 83ec9ede89..cc8332ef68 100644 --- a/src/backend/executor/nodeBitmapIndexscan.c +++ b/src/backend/executor/nodeBitmapIndexscan.c @@ -263,6 +263,8 @@ ExecInitBitmapIndexScan(BitmapIndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->biss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeBitmapOr.c b/src/backend/executor/nodeBitmapOr.c index 0bf8af9652..babad1b4b2 100644 --- a/src/backend/executor/nodeBitmapOr.c +++ b/src/backend/executor/nodeBitmapOr.c @@ -58,6 +58,7 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) BitmapOrState *bitmaporstate = makeNode(BitmapOrState); PlanState **bitmapplanstates; int nplans; + int ninited = 0; int i; ListCell *l; Plan *initNode; @@ -78,8 +79,6 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) bitmaporstate->ps.plan = (Plan *) node; bitmaporstate->ps.state = estate; bitmaporstate->ps.ExecProcNode = ExecBitmapOr; - bitmaporstate->bitmapplans = bitmapplanstates; - bitmaporstate->nplans = nplans; /* * call ExecInitNode on each of the plans to be executed and save the @@ -90,6 +89,9 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) { initNode = (Plan *) lfirst(l); bitmapplanstates[i] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; i++; } @@ -100,6 +102,10 @@ ExecInitBitmapOr(BitmapOr *node, EState *estate, int eflags) * ExecQual or ExecProject. They don't need any tuple slots either. */ +early_exit: + bitmaporstate->bitmapplans = bitmapplanstates; + bitmaporstate->nplans = ninited; + return bitmaporstate; } diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c index cc4c4243e2..eed5b75a4f 100644 --- a/src/backend/executor/nodeCtescan.c +++ b/src/backend/executor/nodeCtescan.c @@ -297,14 +297,16 @@ ExecEndCteScan(CteScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * If I am the leader, free the tuplestore. */ if (node->leader == node) { - tuplestore_end(node->cte_table); + if (node->cte_table) + tuplestore_end(node->cte_table); node->cte_table = NULL; } } diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c index bd42c65b29..544593ccaf 100644 --- a/src/backend/executor/nodeCustom.c +++ b/src/backend/executor/nodeCustom.c @@ -61,6 +61,8 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags) if (scanrelid > 0) { scan_rel = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return css; css->ss.ss_currentRelation = scan_rel; } @@ -127,6 +129,11 @@ ExecCustomScan(PlanState *pstate) void ExecEndCustomScan(CustomScanState *node) { + /* + * XXX - BeginCustomScan() may not have occurred if ExecInitCustomScan() + * hit the early exit case. Perhaps we should document that the custom + * scan provider should be ready to encounter such a situation. + */ Assert(node->methods->EndCustomScan != NULL); node->methods->EndCustomScan(node); @@ -134,8 +141,10 @@ ExecEndCustomScan(CustomScanState *node) ExecFreeExprContext(&node->ss.ps); /* Clean out the tuple table */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } void diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c index c2139acca0..aa54f60127 100644 --- a/src/backend/executor/nodeForeignscan.c +++ b/src/backend/executor/nodeForeignscan.c @@ -173,6 +173,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (scanrelid > 0) { currentRelation = ExecOpenScanRelation(estate, scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; scanstate->ss.ss_currentRelation = currentRelation; fdwroutine = GetFdwRoutineForRelation(currentRelation, true); } @@ -264,6 +266,8 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) if (outerPlan(node)) outerPlanState(scanstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* * Tell the FDW to initialize the scan. @@ -300,14 +304,23 @@ ExecEndForeignScan(ForeignScanState *node) ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; EState *estate = node->ss.ps.state; - /* Let the FDW shut down */ - if (plan->operation != CMD_SELECT) + /* + * Let the FDW shut down if needed. + * + * XXX - BeginDirectModify()/BeginForeignScan() may not have been called + * if ExecInitForeignScan() returned early due to plan being invalidated + * upon taking a lock on the foreign table. + */ + if (node->fdw_state) { - if (estate->es_epq_active == NULL) - node->fdwroutine->EndDirectModify(node); + if (plan->operation != CMD_SELECT) + { + if (estate->es_epq_active == NULL) + node->fdwroutine->EndDirectModify(node); + } + else + node->fdwroutine->EndForeignScan(node); } - else - node->fdwroutine->EndForeignScan(node); /* Shut down any outer plan. */ if (outerPlanState(node)) @@ -319,7 +332,8 @@ ExecEndForeignScan(ForeignScanState *node) /* clean out the tuple table */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c index dd06ef8aee..792ecda4a9 100644 --- a/src/backend/executor/nodeFunctionscan.c +++ b/src/backend/executor/nodeFunctionscan.c @@ -533,7 +533,8 @@ ExecEndFunctionScan(FunctionScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release slots and tuplestore resources diff --git a/src/backend/executor/nodeGather.c b/src/backend/executor/nodeGather.c index 307fc10eea..365d3af3e4 100644 --- a/src/backend/executor/nodeGather.c +++ b/src/backend/executor/nodeGather.c @@ -89,6 +89,8 @@ ExecInitGather(Gather *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gatherstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gatherstate; tupDesc = ExecGetResultType(outerPlanState(gatherstate)); /* diff --git a/src/backend/executor/nodeGatherMerge.c b/src/backend/executor/nodeGatherMerge.c index 9d5e1a46e9..8d2809f079 100644 --- a/src/backend/executor/nodeGatherMerge.c +++ b/src/backend/executor/nodeGatherMerge.c @@ -108,6 +108,8 @@ ExecInitGatherMerge(GatherMerge *node, EState *estate, int eflags) */ outerNode = outerPlan(node); outerPlanState(gm_state) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return gm_state; /* * Leader may access ExecProcNode result directly (if diff --git a/src/backend/executor/nodeGroup.c b/src/backend/executor/nodeGroup.c index 25a1618952..e0832bb778 100644 --- a/src/backend/executor/nodeGroup.c +++ b/src/backend/executor/nodeGroup.c @@ -185,6 +185,8 @@ ExecInitGroup(Group *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(grpstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return grpstate; /* * Initialize scan slot and type. @@ -231,7 +233,8 @@ ExecEndGroup(GroupState *node) ExecFreeExprContext(&node->ss.ps); /* clean up tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c index 748c9b0024..891bcee919 100644 --- a/src/backend/executor/nodeHash.c +++ b/src/backend/executor/nodeHash.c @@ -386,6 +386,8 @@ ExecInitHash(Hash *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(hashstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return hashstate; /* * initialize our result slot and type. No need to build projection diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c index f189fb4d28..93ce0c8be0 100644 --- a/src/backend/executor/nodeHashjoin.c +++ b/src/backend/executor/nodeHashjoin.c @@ -691,8 +691,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags) hashNode = (Hash *) innerPlan(node); outerPlanState(hjstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; outerDesc = ExecGetResultType(outerPlanState(hjstate)); innerPlanState(hjstate) = ExecInitNode((Plan *) hashNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return hjstate; innerDesc = ExecGetResultType(innerPlanState(hjstate)); /* @@ -813,9 +817,12 @@ ExecEndHashJoin(HashJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->hj_OuterTupleSlot); - ExecClearTuple(node->hj_HashTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->hj_OuterTupleSlot) + ExecClearTuple(node->hj_OuterTupleSlot); + if (node->hj_HashTupleSlot) + ExecClearTuple(node->hj_HashTupleSlot); /* * clean up subtrees diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c index 12bc22f33c..6b2da56044 100644 --- a/src/backend/executor/nodeIncrementalSort.c +++ b/src/backend/executor/nodeIncrementalSort.c @@ -1041,6 +1041,8 @@ ExecInitIncrementalSort(IncrementalSort *node, EState *estate, int eflags) * nodes may be able to do something more useful. */ outerPlanState(incrsortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return incrsortstate; /* * Initialize scan slot and type. @@ -1080,12 +1082,16 @@ ExecEndIncrementalSort(IncrementalSortState *node) SO_printf("ExecEndIncrementalSort: shutting down sort node\n"); /* clean out the scan tuple */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* must drop standalone tuple slots from outer node */ - ExecDropSingleTupleTableSlot(node->group_pivot); - ExecDropSingleTupleTableSlot(node->transfer_tuple); + if (node->group_pivot) + ExecDropSingleTupleTableSlot(node->group_pivot); + if (node->transfer_tuple) + ExecDropSingleTupleTableSlot(node->transfer_tuple); /* * Release tuplesort resources. diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index 0b43a9b969..b60a086464 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -394,7 +394,8 @@ ExecEndIndexOnlyScan(IndexOnlyScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if(node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -512,6 +513,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -565,6 +568,8 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->ioss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c index 4540c7781d..628c233919 100644 --- a/src/backend/executor/nodeIndexscan.c +++ b/src/backend/executor/nodeIndexscan.c @@ -808,7 +808,8 @@ ExecEndIndexScan(IndexScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close the index relation (no-op if we didn't open it) @@ -925,6 +926,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return indexstate; indexstate->ss.ss_currentRelation = currentRelation; indexstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ @@ -970,6 +973,8 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags) /* Open the index relation. */ lockmode = exec_rt_fetch(node->scan.scanrelid, estate)->rellockmode; indexstate->iss_RelationDesc = index_open(node->indexid, lockmode); + if (!ExecPlanStillValid(estate)) + return indexstate; /* * Initialize index-specific scan state diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c index 425fbfc405..2fcbde74ed 100644 --- a/src/backend/executor/nodeLimit.c +++ b/src/backend/executor/nodeLimit.c @@ -476,6 +476,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(limitstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return limitstate; /* * initialize child expressions diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c index 407414fc0c..3a8aa2b5a4 100644 --- a/src/backend/executor/nodeLockRows.c +++ b/src/backend/executor/nodeLockRows.c @@ -323,6 +323,8 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(lrstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return lrstate; /* node returns unmodified slots from the outer plan */ lrstate->ps.resultopsset = true; diff --git a/src/backend/executor/nodeMaterial.c b/src/backend/executor/nodeMaterial.c index 09632678b0..f146ebb1d7 100644 --- a/src/backend/executor/nodeMaterial.c +++ b/src/backend/executor/nodeMaterial.c @@ -214,6 +214,8 @@ ExecInitMaterial(Material *node, EState *estate, int eflags) outerPlan = outerPlan(node); outerPlanState(matstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return matstate; /* * Initialize result type and slot. No need to initialize projection info @@ -242,7 +244,8 @@ ExecEndMaterial(MaterialState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeMemoize.c b/src/backend/executor/nodeMemoize.c index 4f04269e26..3003ee1e5c 100644 --- a/src/backend/executor/nodeMemoize.c +++ b/src/backend/executor/nodeMemoize.c @@ -938,6 +938,8 @@ ExecInitMemoize(Memoize *node, EState *estate, int eflags) outerNode = outerPlan(node); outerPlanState(mstate) = ExecInitNode(outerNode, estate, eflags); + if (!ExecPlanStillValid(estate)) + return mstate; /* * Initialize return slot and type. No need to initialize projection info @@ -1043,6 +1045,7 @@ ExecEndMemoize(MemoizeState *node) { #ifdef USE_ASSERT_CHECKING /* Validate the memory accounting code is correct in assert builds. */ + if (node->hashtable) { int count; uint64 mem = 0; @@ -1089,11 +1092,14 @@ ExecEndMemoize(MemoizeState *node) } /* Remove the cache context */ - MemoryContextDelete(node->tableContext); + if (node->tableContext) + MemoryContextDelete(node->tableContext); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to cache result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * free exprcontext diff --git a/src/backend/executor/nodeMergeAppend.c b/src/backend/executor/nodeMergeAppend.c index 399b39c598..40bba35499 100644 --- a/src/backend/executor/nodeMergeAppend.c +++ b/src/backend/executor/nodeMergeAppend.c @@ -65,9 +65,10 @@ MergeAppendState * ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) { MergeAppendState *mergestate = makeNode(MergeAppendState); - PlanState **mergeplanstates; + PlanState **mergeplanstates = NULL; Bitmapset *validsubplans; int nplans; + int ninited = 0; int i, j; @@ -81,6 +82,15 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) mergestate->ps.state = estate; mergestate->ps.ExecProcNode = ExecMergeAppend; + /* + * Lock non-leaf partitions. In the pruning case, some of these locks + * will be retaken when the partition will be opened for pruning, but it + * does not seem worthwhile to spend cycles to filter those out here. + */ + ExecLockAppendNonLeafRelations(estate, node->allpartrelids); + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* If run-time partition pruning is enabled, then set that up now */ if (node->part_prune_index >= 0) { @@ -96,6 +106,8 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) node->part_prune_index, node->apprelids, &validsubplans); + if (!ExecPlanStillValid(estate)) + return mergestate; mergestate->ms_prune_state = prunestate; nplans = bms_num_members(validsubplans); @@ -122,8 +134,6 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) } mergeplanstates = (PlanState **) palloc(nplans * sizeof(PlanState *)); - mergestate->mergeplans = mergeplanstates; - mergestate->ms_nplans = nplans; mergestate->ms_slots = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans); mergestate->ms_heap = binaryheap_allocate(nplans, heap_compare_slots, @@ -152,6 +162,9 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) Plan *initNode = (Plan *) list_nth(node->mergeplans, i); mergeplanstates[j++] = ExecInitNode(initNode, estate, eflags); + ninited++; + if (!ExecPlanStillValid(estate)) + goto early_exit; } mergestate->ps.ps_ProjInfo = NULL; @@ -188,6 +201,10 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags) */ mergestate->ms_initialized = false; +early_exit: + mergestate->mergeplans = mergeplanstates; + mergestate->ms_nplans = ninited; + return mergestate; } diff --git a/src/backend/executor/nodeMergejoin.c b/src/backend/executor/nodeMergejoin.c index 809aa215c6..968be05568 100644 --- a/src/backend/executor/nodeMergejoin.c +++ b/src/backend/executor/nodeMergejoin.c @@ -1482,11 +1482,15 @@ ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags) mergestate->mj_SkipMarkRestore = node->skip_mark_restore; outerPlanState(mergestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return mergestate; outerDesc = ExecGetResultType(outerPlanState(mergestate)); innerPlanState(mergestate) = ExecInitNode(innerPlan(node), estate, mergestate->mj_SkipMarkRestore ? eflags : (eflags | EXEC_FLAG_MARK)); + if (!ExecPlanStillValid(estate)) + return mergestate; innerDesc = ExecGetResultType(innerPlanState(mergestate)); /* @@ -1642,8 +1646,10 @@ ExecEndMergeJoin(MergeJoinState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); - ExecClearTuple(node->mj_MarkedTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->mj_MarkedTupleSlot) + ExecClearTuple(node->mj_MarkedTupleSlot); /* * shut down the subplans diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c index e350375681..8a70543326 100644 --- a/src/backend/executor/nodeModifyTable.c +++ b/src/backend/executor/nodeModifyTable.c @@ -3900,6 +3900,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) Plan *subplan = outerPlan(node); CmdType operation = node->operation; int nrels = list_length(node->resultRelations); + int ninited = 0; ResultRelInfo *resultRelInfo; List *arowmarks; ListCell *l; @@ -3921,7 +3922,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) mtstate->canSetTag = node->canSetTag; mtstate->mt_done = false; - mtstate->mt_nrels = nrels; mtstate->resultRelInfo = (ResultRelInfo *) palloc(nrels * sizeof(ResultRelInfo)); @@ -3956,6 +3956,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) linitial_int(node->resultRelations)); } + if (!ExecPlanStillValid(estate)) + goto early_exit; + /* set up epqstate with dummy subplan data for the moment */ EvalPlanQualInit(&mtstate->mt_epqstate, estate, NULL, NIL, node->epqParam); mtstate->fireBSTriggers = true; @@ -3982,6 +3985,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) if (resultRelInfo != mtstate->rootResultRelInfo) { ExecInitResultRelation(estate, resultRelInfo, resultRelation); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * For child result relations, store the root result relation @@ -4009,11 +4014,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) * Now we may initialize the subplan. */ outerPlanState(mtstate) = ExecInitNode(subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + goto early_exit; /* * Do additional per-result-relation initialization. */ - for (i = 0; i < nrels; i++) + for (i = 0; i < nrels; i++, ninited++) { resultRelInfo = &mtstate->resultRelInfo[i]; @@ -4362,6 +4369,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags) estate->es_auxmodifytables = lcons(mtstate, estate->es_auxmodifytables); +early_exit: + mtstate->mt_nrels = ninited; return mtstate; } diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c index 46832ad82f..1f92c43d3b 100644 --- a/src/backend/executor/nodeNamedtuplestorescan.c +++ b/src/backend/executor/nodeNamedtuplestorescan.c @@ -174,7 +174,8 @@ ExecEndNamedTuplestoreScan(NamedTuplestoreScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeNestloop.c b/src/backend/executor/nodeNestloop.c index b3d52e69ec..deda0c2559 100644 --- a/src/backend/executor/nodeNestloop.c +++ b/src/backend/executor/nodeNestloop.c @@ -295,11 +295,15 @@ ExecInitNestLoop(NestLoop *node, EState *estate, int eflags) * values. */ outerPlanState(nlstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; if (node->nestParams == NIL) eflags |= EXEC_FLAG_REWIND; else eflags &= ~EXEC_FLAG_REWIND; innerPlanState(nlstate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return nlstate; /* * Initialize result slot, type and projection. @@ -372,7 +376,8 @@ ExecEndNestLoop(NestLoopState *node) /* * clean out the tuple table */ - ExecClearTuple(node->js.ps.ps_ResultTupleSlot); + if (node->js.ps.ps_ResultTupleSlot) + ExecClearTuple(node->js.ps.ps_ResultTupleSlot); /* * close down subplans diff --git a/src/backend/executor/nodeProjectSet.c b/src/backend/executor/nodeProjectSet.c index f6ff3dc44c..85d20c4680 100644 --- a/src/backend/executor/nodeProjectSet.c +++ b/src/backend/executor/nodeProjectSet.c @@ -247,6 +247,8 @@ ExecInitProjectSet(ProjectSet *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(state) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return state; /* * we don't use inner plan @@ -328,7 +330,8 @@ ExecEndProjectSet(ProjectSetState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeRecursiveunion.c b/src/backend/executor/nodeRecursiveunion.c index e781003934..967fe4f287 100644 --- a/src/backend/executor/nodeRecursiveunion.c +++ b/src/backend/executor/nodeRecursiveunion.c @@ -244,7 +244,11 @@ ExecInitRecursiveUnion(RecursiveUnion *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(rustate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; innerPlanState(rustate) = ExecInitNode(innerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return rustate; /* * If hashing, precompute fmgr lookup data for inner loop, and create the diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c index 4219712d30..c549b684a3 100644 --- a/src/backend/executor/nodeResult.c +++ b/src/backend/executor/nodeResult.c @@ -208,6 +208,8 @@ ExecInitResult(Result *node, EState *estate, int eflags) * initialize child nodes */ outerPlanState(resstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return resstate; /* * we don't use inner plan @@ -248,7 +250,8 @@ ExecEndResult(ResultState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* * shut down subplans diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c index d7e22b1dbb..b3bc9b1f77 100644 --- a/src/backend/executor/nodeSamplescan.c +++ b/src/backend/executor/nodeSamplescan.c @@ -125,6 +125,8 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* we won't set up the HeapScanDesc till later */ scanstate->ss.ss_currentScanDesc = NULL; @@ -198,7 +200,8 @@ ExecEndSampleScan(SampleScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c index 4da0f28f7b..e7ca19ee4e 100644 --- a/src/backend/executor/nodeSeqscan.c +++ b/src/backend/executor/nodeSeqscan.c @@ -153,6 +153,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags) ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return scanstate; /* and create slot with the appropriate rowtype */ ExecInitScanTupleSlot(estate, &scanstate->ss, @@ -200,7 +202,8 @@ ExecEndSeqScan(SeqScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close heap scan diff --git a/src/backend/executor/nodeSetOp.c b/src/backend/executor/nodeSetOp.c index 4bc2406b89..95950a5c20 100644 --- a/src/backend/executor/nodeSetOp.c +++ b/src/backend/executor/nodeSetOp.c @@ -528,6 +528,8 @@ ExecInitSetOp(SetOp *node, EState *estate, int eflags) if (node->strategy == SETOP_HASHED) eflags &= ~EXEC_FLAG_REWIND; outerPlanState(setopstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return setopstate; outerDesc = ExecGetResultType(outerPlanState(setopstate)); /* @@ -583,7 +585,8 @@ void ExecEndSetOp(SetOpState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); /* free subsidiary stuff including hashtable */ if (node->tableContext) diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c index c6c72c6e67..89fef86aba 100644 --- a/src/backend/executor/nodeSort.c +++ b/src/backend/executor/nodeSort.c @@ -263,6 +263,8 @@ ExecInitSort(Sort *node, EState *estate, int eflags) eflags &= ~(EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK); outerPlanState(sortstate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return sortstate; /* * Initialize scan slot and type. @@ -306,9 +308,11 @@ ExecEndSort(SortState *node) /* * clean out the tuple table */ - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* must drop pointer to sort result tuple */ - ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); + if (node->ss.ps.ps_ResultTupleSlot) + ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); /* * Release tuplesort resources diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c index 42471bfc04..9b8cddc89f 100644 --- a/src/backend/executor/nodeSubqueryscan.c +++ b/src/backend/executor/nodeSubqueryscan.c @@ -124,6 +124,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags) * initialize subquery */ subquerystate->subplan = ExecInitNode(node->subplan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return subquerystate; /* * Initialize scan slot and type (needed by ExecAssignScanProjectionInfo) @@ -177,7 +179,8 @@ ExecEndSubqueryScan(SubqueryScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * close down subquery diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c index 0c6c912778..d7536953f1 100644 --- a/src/backend/executor/nodeTableFuncscan.c +++ b/src/backend/executor/nodeTableFuncscan.c @@ -223,7 +223,8 @@ ExecEndTableFuncScan(TableFuncScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); /* * Release tuplestore resources diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c index 2124c55ef5..1ae451d7a6 100644 --- a/src/backend/executor/nodeTidrangescan.c +++ b/src/backend/executor/nodeTidrangescan.c @@ -342,7 +342,8 @@ ExecEndTidRangeScan(TidRangeScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -386,6 +387,8 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidrangestate; tidrangestate->ss.ss_currentRelation = currentRelation; tidrangestate->ss.ss_currentScanDesc = NULL; /* no table scan here */ diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c index 862bd0330b..9fe76b1c60 100644 --- a/src/backend/executor/nodeTidscan.c +++ b/src/backend/executor/nodeTidscan.c @@ -483,7 +483,8 @@ ExecEndTidScan(TidScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- @@ -529,6 +530,8 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags) * open the scan relation */ currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid, eflags); + if (!ExecPlanStillValid(estate)) + return tidstate; tidstate->ss.ss_currentRelation = currentRelation; tidstate->ss.ss_currentScanDesc = NULL; /* no heap scan here */ diff --git a/src/backend/executor/nodeUnique.c b/src/backend/executor/nodeUnique.c index 45035d74fa..69f23b02c6 100644 --- a/src/backend/executor/nodeUnique.c +++ b/src/backend/executor/nodeUnique.c @@ -136,6 +136,8 @@ ExecInitUnique(Unique *node, EState *estate, int eflags) * then initialize outer plan */ outerPlanState(uniquestate) = ExecInitNode(outerPlan(node), estate, eflags); + if (!ExecPlanStillValid(estate)) + return uniquestate; /* * Initialize result slot and type. Unique nodes do no projections, so @@ -169,7 +171,8 @@ void ExecEndUnique(UniqueState *node) { /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); ExecFreeExprContext(&node->ps); diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c index 32ace63017..f5dedbab63 100644 --- a/src/backend/executor/nodeValuesscan.c +++ b/src/backend/executor/nodeValuesscan.c @@ -340,7 +340,8 @@ ExecEndValuesScan(ValuesScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/nodeWindowAgg.c b/src/backend/executor/nodeWindowAgg.c index 7c07fb0684..616bb97675 100644 --- a/src/backend/executor/nodeWindowAgg.c +++ b/src/backend/executor/nodeWindowAgg.c @@ -1334,7 +1334,7 @@ release_partition(WindowAggState *winstate) WindowStatePerFunc perfuncstate = &(winstate->perfunc[i]); /* Release any partition-local state of this window function */ - if (perfuncstate->winobj) + if (perfuncstate && perfuncstate->winobj) perfuncstate->winobj->localmem = NULL; } @@ -1344,12 +1344,17 @@ release_partition(WindowAggState *winstate) * any aggregate temp data). We don't rely on retail pfree because some * aggregates might have allocated data we don't have direct pointers to. */ - MemoryContextResetAndDeleteChildren(winstate->partcontext); - MemoryContextResetAndDeleteChildren(winstate->aggcontext); - for (i = 0; i < winstate->numaggs; i++) + if (winstate->partcontext) + MemoryContextResetAndDeleteChildren(winstate->partcontext); + if (winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->aggcontext); + if (winstate->peragg) { - if (winstate->peragg[i].aggcontext != winstate->aggcontext) - MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + for (i = 0; i < winstate->numaggs; i++) + { + if (winstate->peragg[i].aggcontext != winstate->aggcontext) + MemoryContextResetAndDeleteChildren(winstate->peragg[i].aggcontext); + } } if (winstate->buffer) @@ -2451,6 +2456,8 @@ ExecInitWindowAgg(WindowAgg *node, EState *estate, int eflags) */ outerPlan = outerPlan(node); outerPlanState(winstate) = ExecInitNode(outerPlan, estate, eflags); + if (!ExecPlanStillValid(estate)) + return winstate; /* * initialize source tuple type (which is also the tuple type that we'll @@ -2679,11 +2686,16 @@ ExecEndWindowAgg(WindowAggState *node) release_partition(node); - ExecClearTuple(node->ss.ss_ScanTupleSlot); - ExecClearTuple(node->first_part_slot); - ExecClearTuple(node->agg_row_slot); - ExecClearTuple(node->temp_slot_1); - ExecClearTuple(node->temp_slot_2); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->first_part_slot) + ExecClearTuple(node->first_part_slot); + if (node->agg_row_slot) + ExecClearTuple(node->agg_row_slot); + if (node->temp_slot_1) + ExecClearTuple(node->temp_slot_1); + if (node->temp_slot_2) + ExecClearTuple(node->temp_slot_2); if (node->framehead_slot) ExecClearTuple(node->framehead_slot); if (node->frametail_slot) @@ -2696,16 +2708,23 @@ ExecEndWindowAgg(WindowAggState *node) node->ss.ps.ps_ExprContext = node->tmpcontext; ExecFreeExprContext(&node->ss.ps); - for (i = 0; i < node->numaggs; i++) + if (node->peragg) { - if (node->peragg[i].aggcontext != node->aggcontext) - MemoryContextDelete(node->peragg[i].aggcontext); + for (i = 0; i < node->numaggs; i++) + { + if (node->peragg[i].aggcontext != node->aggcontext) + MemoryContextDelete(node->peragg[i].aggcontext); + } } - MemoryContextDelete(node->partcontext); - MemoryContextDelete(node->aggcontext); + if (node->partcontext) + MemoryContextDelete(node->partcontext); + if (node->aggcontext) + MemoryContextDelete(node->aggcontext); - pfree(node->perfunc); - pfree(node->peragg); + if (node->perfunc) + pfree(node->perfunc); + if (node->peragg) + pfree(node->peragg); outerPlan = outerPlanState(node); ExecEndNode(outerPlan); diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c index 0c13448236..d70c6afde3 100644 --- a/src/backend/executor/nodeWorktablescan.c +++ b/src/backend/executor/nodeWorktablescan.c @@ -200,7 +200,8 @@ ExecEndWorkTableScan(WorkTableScanState *node) */ if (node->ss.ps.ps_ResultTupleSlot) ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); - ExecClearTuple(node->ss.ss_ScanTupleSlot); + if (node->ss.ss_ScanTupleSlot) + ExecClearTuple(node->ss.ss_ScanTupleSlot); } /* ---------------------------------------------------------------- diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index e3a170c38b..26a9ea342a 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -71,7 +71,7 @@ static int _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, static ParamListInfo _SPI_convert_params(int nargs, Oid *argtypes, Datum *Values, const char *Nulls); -static int _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount); +static int _SPI_pquery(QueryDesc *queryDesc, uint64 tcount); static void _SPI_error_callback(void *arg); @@ -1623,6 +1623,7 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, _SPI_current->processed = 0; _SPI_current->tuptable = NULL; +replan: /* Create the portal */ if (name == NULL || name[0] == '\0') { @@ -1766,7 +1767,10 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, } /* - * Start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, paramLI, 0, snapshot); @@ -1775,6 +1779,12 @@ SPI_cursor_open_internal(const char *name, SPIPlanPtr plan, /* Pop the error context stack */ error_context_stack = spierrcontext.previous; + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* Pop the SPI stack */ _SPI_end_call(true); @@ -2552,6 +2562,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, * Replan if needed, and increment plan refcount. If it's a saved * plan, the refcount must be backed by the plan_owner. */ +replan: cplan = GetCachedPlan(plansource, options->params, plan_owner, _SPI_current->queryEnv); @@ -2661,6 +2672,7 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, { QueryDesc *qdesc; Snapshot snap; + int eflags; if (ActiveSnapshotSet()) snap = GetActiveSnapshot(); @@ -2668,14 +2680,36 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options, snap = InvalidSnapshot; qdesc = CreateQueryDesc(stmt, + cplan, plansource->query_string, snap, crosscheck_snapshot, dest, options->params, _SPI_current->queryEnv, 0); - res = _SPI_pquery(qdesc, fire_triggers, - canSetTag ? options->tcount : 0); + + /* Select execution options */ + if (fire_triggers) + eflags = 0; /* default run-to-completion flags */ + else + eflags = EXEC_FLAG_SKIP_TRIGGERS; + + /* Take locks if using a CachedPlan */ + if (qdesc->cplan) + eflags |= EXEC_FLAG_GET_LOCKS; + + ExecutorStart(qdesc, eflags); + if (!qdesc->plan_valid) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + FreeQueryDesc(qdesc); + Assert(cplan); + ReleaseCachedPlan(cplan, plan_owner); + goto replan; + } + + res = _SPI_pquery(qdesc, canSetTag ? options->tcount : 0); FreeQueryDesc(qdesc); } else @@ -2850,10 +2884,9 @@ _SPI_convert_params(int nargs, Oid *argtypes, } static int -_SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) +_SPI_pquery(QueryDesc *queryDesc, uint64 tcount) { int operation = queryDesc->operation; - int eflags; int res; switch (operation) @@ -2897,14 +2930,6 @@ _SPI_pquery(QueryDesc *queryDesc, bool fire_triggers, uint64 tcount) ResetUsage(); #endif - /* Select execution options */ - if (fire_triggers) - eflags = 0; /* default run-to-completion flags */ - else - eflags = EXEC_FLAG_SKIP_TRIGGERS; - - ExecutorStart(queryDesc, eflags); - ExecutorRun(queryDesc, ForwardScanDirection, tcount, true); _SPI_current->processed = queryDesc->estate->es_processed; diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index ba00b99249..955286513d 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @@ -513,6 +513,7 @@ _outRangeTblEntry(StringInfo str, const RangeTblEntry *node) WRITE_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ WRITE_OID_FIELD(relid); + WRITE_CHAR_FIELD(relkind); WRITE_INT_FIELD(rellockmode); WRITE_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c index 597e5b3ea8..a136ae1d60 100644 --- a/src/backend/nodes/readfuncs.c +++ b/src/backend/nodes/readfuncs.c @@ -503,6 +503,7 @@ _readRangeTblEntry(void) READ_BOOL_FIELD(security_barrier); /* we re-use these RELATION fields, too: */ READ_OID_FIELD(relid); + READ_CHAR_FIELD(relkind); READ_INT_FIELD(rellockmode); READ_UINT_FIELD(perminfoindex); break; diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index 62b3ec96cc..5f3ffd98af 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -527,6 +527,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions, result->partPruneInfos = glob->partPruneInfos; result->rtable = glob->finalrtable; result->permInfos = glob->finalrteperminfos; + result->viewRelations = glob->viewRelations; result->resultRelations = glob->resultRelations; result->appendRelations = glob->appendRelations; result->subplans = glob->subplans; diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c index 5cc8366af6..f13240bf33 100644 --- a/src/backend/optimizer/plan/setrefs.c +++ b/src/backend/optimizer/plan/setrefs.c @@ -16,6 +16,7 @@ #include "postgres.h" #include "access/transam.h" +#include "catalog/pg_class.h" #include "catalog/pg_type.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" @@ -604,6 +605,10 @@ add_rte_to_flat_rtable(PlannerGlobal *glob, List *rteperminfos, (newrte->rtekind == RTE_SUBQUERY && OidIsValid(newrte->relid))) glob->relationOids = lappend_oid(glob->relationOids, newrte->relid); + if (newrte->relkind == RELKIND_VIEW) + glob->viewRelations = lappend_int(glob->viewRelations, + list_length(glob->finalrtable)); + /* * Add a copy of the RTEPermissionInfo, if any, corresponding to this RTE * to the flattened global list. diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c index 980dc1816f..1631c8b993 100644 --- a/src/backend/rewrite/rewriteHandler.c +++ b/src/backend/rewrite/rewriteHandler.c @@ -1849,11 +1849,10 @@ ApplyRetrieveRule(Query *parsetree, /* * Clear fields that should not be set in a subquery RTE. Note that we - * leave the relid, rellockmode, and perminfoindex fields set, so that the - * view relation can be appropriately locked before execution and its - * permissions checked. + * leave the relid, relkind, rellockmode, and perminfoindex fields set, + * so that the view relation can be appropriately locked before execution + * and its permissions checked. */ - rte->relkind = 0; rte->tablesample = NULL; rte->inh = false; /* must not be set for a subquery */ diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index ee9b89a672..c807e9cdcc 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -27,6 +27,7 @@ #include "storage/procarray.h" #include "storage/sinvaladt.h" #include "utils/inval.h" +#include "utils/lsyscache.h" /* @@ -364,6 +365,50 @@ CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger) return false; } +/* + * CheckRelLockedByMe + * + * Returns true if current transaction holds a lock on the given relation of + * mode 'lockmode'. If 'orstronger' is true, a stronger lockmode is also OK. + * ("Stronger" is defined as "numerically higher", which is a bit + * semantically dubious but is OK for the purposes we use this for.) + */ +bool +CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger) +{ + Oid dbId = get_rel_relisshared(relid) ? InvalidOid : MyDatabaseId; + LOCKTAG tag; + + SET_LOCKTAG_RELATION(tag, dbId, relid); + + if (LockHeldByMe(&tag, lockmode)) + return true; + + if (orstronger) + { + LOCKMODE slockmode; + + for (slockmode = lockmode + 1; + slockmode <= MaxLockMode; + slockmode++) + { + if (LockHeldByMe(&tag, slockmode)) + { +#ifdef NOT_USED + /* Sometimes this might be useful for debugging purposes */ + elog(WARNING, "lock mode %s substituted for %s on relation %s", + GetLockmodeName(tag.locktag_lockmethodid, slockmode), + GetLockmodeName(tag.locktag_lockmethodid, lockmode), + RelationGetRelationName(relation)); +#endif + return true; + } + } + } + + return false; +} + /* * LockHasWaitersRelation * diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index cab709b07b..6d0ea07801 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1199,6 +1199,7 @@ exec_simple_query(const char *query_string) * Start the portal. No parameters here. */ PortalStart(portal, NULL, 0, InvalidSnapshot); + Assert(portal->plan_valid); /* * Select the appropriate output format: text unless we are doing a @@ -1703,6 +1704,7 @@ exec_bind_message(StringInfo input_message) "commands ignored until end of transaction block"), errdetail_abort())); +replan: /* * Create the portal. Allow silent replacement of an existing portal only * if the unnamed portal is specified. @@ -1994,10 +1996,19 @@ exec_bind_message(StringInfo input_message) PopActiveSnapshot(); /* - * And we're ready to start portal execution. + * Start portal execution. If the portal contains a cached plan, it must + * be recreated if portal->plan_valid is false which tells that the cached + * plan was found to have been invalidated when initializing one of the + * plan trees contained in it. */ PortalStart(portal, params, 0, InvalidSnapshot); + if (!portal->plan_valid) + { + PortalDrop(portal, false); + goto replan; + } + /* * Apply the result format requests to the portal. */ diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c index 5f0248acc5..c93a950d7f 100644 --- a/src/backend/tcop/pquery.c +++ b/src/backend/tcop/pquery.c @@ -19,6 +19,7 @@ #include "access/xact.h" #include "commands/prepare.h" +#include "executor/execdesc.h" #include "executor/tstoreReceiver.h" #include "miscadmin.h" #include "pg_trace.h" @@ -35,12 +36,6 @@ Portal ActivePortal = NULL; -static void ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc); static void FillPortalStore(Portal portal, bool isTopLevel); static uint64 RunFromStore(Portal portal, ScanDirection direction, uint64 count, DestReceiver *dest); @@ -65,6 +60,7 @@ static void DoPortalRewind(Portal portal); */ QueryDesc * CreateQueryDesc(PlannedStmt *plannedstmt, + CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, @@ -77,6 +73,7 @@ CreateQueryDesc(PlannedStmt *plannedstmt, qd->operation = plannedstmt->commandType; /* operation */ qd->plannedstmt = plannedstmt; /* plan */ + qd->cplan = cplan; /* CachedPlan, if plan is from one */ qd->sourceText = sourceText; /* query text */ qd->snapshot = RegisterSnapshot(snapshot); /* snapshot */ /* RI check snapshot */ @@ -116,86 +113,6 @@ FreeQueryDesc(QueryDesc *qdesc) } -/* - * ProcessQuery - * Execute a single plannable query within a PORTAL_MULTI_QUERY, - * PORTAL_ONE_RETURNING, or PORTAL_ONE_MOD_WITH portal - * - * plan: the plan tree for the query - * sourceText: the source text of the query - * params: any parameters needed - * dest: where to send results - * qc: where to store the command completion status data. - * - * qc may be NULL if caller doesn't want a status string. - * - * Must be called in a memory context that will be reset or deleted on - * error; otherwise the executor's memory usage will be leaked. - */ -static void -ProcessQuery(PlannedStmt *plan, - const char *sourceText, - ParamListInfo params, - QueryEnvironment *queryEnv, - DestReceiver *dest, - QueryCompletion *qc) -{ - QueryDesc *queryDesc; - - /* - * Create the QueryDesc object - */ - queryDesc = CreateQueryDesc(plan, sourceText, - GetActiveSnapshot(), InvalidSnapshot, - dest, params, queryEnv, 0); - - /* - * Call ExecutorStart to prepare the plan for execution - */ - ExecutorStart(queryDesc, 0); - - /* - * Run the plan to completion. - */ - ExecutorRun(queryDesc, ForwardScanDirection, 0L, true); - - /* - * Build command completion status data, if caller wants one. - */ - if (qc) - { - switch (queryDesc->operation) - { - case CMD_SELECT: - SetQueryCompletion(qc, CMDTAG_SELECT, queryDesc->estate->es_processed); - break; - case CMD_INSERT: - SetQueryCompletion(qc, CMDTAG_INSERT, queryDesc->estate->es_processed); - break; - case CMD_UPDATE: - SetQueryCompletion(qc, CMDTAG_UPDATE, queryDesc->estate->es_processed); - break; - case CMD_DELETE: - SetQueryCompletion(qc, CMDTAG_DELETE, queryDesc->estate->es_processed); - break; - case CMD_MERGE: - SetQueryCompletion(qc, CMDTAG_MERGE, queryDesc->estate->es_processed); - break; - default: - SetQueryCompletion(qc, CMDTAG_UNKNOWN, queryDesc->estate->es_processed); - break; - } - } - - /* - * Now, we close down all the scans and free allocated resources. - */ - ExecutorFinish(queryDesc); - ExecutorEnd(queryDesc); - - FreeQueryDesc(queryDesc); -} - /* * ChoosePortalStrategy * Select portal execution strategy given the intended statement list. @@ -427,7 +344,8 @@ FetchStatementTargetList(Node *stmt) * to be used for cursors). * * On return, portal is ready to accept PortalRun() calls, and the result - * tupdesc (if any) is known. + * tupdesc (if any) is known, unless portal->plan_valid is set to false, in + * which case, the caller must retry after generating a new CachedPlan. */ void PortalStart(Portal portal, ParamListInfo params, @@ -435,7 +353,6 @@ PortalStart(Portal portal, ParamListInfo params, { Portal saveActivePortal; ResourceOwner saveResourceOwner; - MemoryContext savePortalContext; MemoryContext oldContext; QueryDesc *queryDesc; int myeflags; @@ -448,15 +365,13 @@ PortalStart(Portal portal, ParamListInfo params, */ saveActivePortal = ActivePortal; saveResourceOwner = CurrentResourceOwner; - savePortalContext = PortalContext; PG_TRY(); { ActivePortal = portal; if (portal->resowner) CurrentResourceOwner = portal->resowner; - PortalContext = portal->portalContext; - oldContext = MemoryContextSwitchTo(PortalContext); + oldContext = MemoryContextSwitchTo(portal->queryContext); /* Must remember portal param list, if any */ portal->portalParams = params; @@ -472,6 +387,8 @@ PortalStart(Portal portal, ParamListInfo params, switch (portal->strategy) { case PORTAL_ONE_SELECT: + case PORTAL_ONE_RETURNING: + case PORTAL_ONE_MOD_WITH: /* Must set snapshot before starting executor. */ if (snapshot) @@ -493,6 +410,7 @@ PortalStart(Portal portal, ParamListInfo params, * the destination to DestNone. */ queryDesc = CreateQueryDesc(linitial_node(PlannedStmt, portal->stmts), + portal->cplan, portal->sourceText, GetActiveSnapshot(), InvalidSnapshot, @@ -501,30 +419,56 @@ PortalStart(Portal portal, ParamListInfo params, portal->queryEnv, 0); + /* Remember for PortalRunMulti(). */ + if (portal->strategy == PORTAL_ONE_RETURNING || + portal->strategy == PORTAL_ONE_MOD_WITH) + portal->qdescs = list_make1(queryDesc); + /* * If it's a scrollable cursor, executor needs to support * REWIND and backwards scan, as well as whatever the caller * might've asked for. */ - if (portal->cursorOptions & CURSOR_OPT_SCROLL) + if (portal->strategy == PORTAL_ONE_SELECT && + (portal->cursorOptions & CURSOR_OPT_SCROLL)) myeflags = eflags | EXEC_FLAG_REWIND | EXEC_FLAG_BACKWARD; else myeflags = eflags; + /* Take locks if using a CachedPlan */ + if (queryDesc->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + /* - * Call ExecutorStart to prepare the plan for execution + * Call ExecutorStart to prepare the plan for execution. A + * cached plan may get invalidated as we're doing that. */ ExecutorStart(queryDesc, myeflags); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + PopActiveSnapshot(); + portal->plan_valid = false; + goto early_exit; + } /* - * This tells PortalCleanup to shut down the executor + * This tells PortalCleanup to shut down the executor, though + * not needed for queries handled by PortalRunMulti(). */ - portal->queryDesc = queryDesc; + if (portal->strategy == PORTAL_ONE_SELECT) + portal->queryDesc = queryDesc; /* - * Remember tuple descriptor (computed by ExecutorStart) + * Remember tuple descriptor (computed by ExecutorStart), + * though make it independent of QueryDesc for queries handled + * by PortalRunMulti(). */ - portal->tupDesc = queryDesc->tupDesc; + if (portal->strategy != PORTAL_ONE_SELECT) + portal->tupDesc = CreateTupleDescCopy(queryDesc->tupDesc); + else + portal->tupDesc = queryDesc->tupDesc; /* * Reset cursor position data to "start of query" @@ -532,33 +476,11 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; PopActiveSnapshot(); break; - case PORTAL_ONE_RETURNING: - case PORTAL_ONE_MOD_WITH: - - /* - * We don't start the executor until we are told to run the - * portal. We do need to set up the result tupdesc. - */ - { - PlannedStmt *pstmt; - - pstmt = PortalGetPrimaryStmt(portal); - portal->tupDesc = - ExecCleanTypeFromTL(pstmt->planTree->targetlist); - } - - /* - * Reset cursor position data to "start of query" - */ - portal->atStart = true; - portal->atEnd = false; /* allow fetches */ - portal->portalPos = 0; - break; - case PORTAL_UTIL_SELECT: /* @@ -578,11 +500,90 @@ PortalStart(Portal portal, ParamListInfo params, portal->atStart = true; portal->atEnd = false; /* allow fetches */ portal->portalPos = 0; + portal->plan_valid = true; break; case PORTAL_MULTI_QUERY: - /* Need do nothing now */ + { + ListCell *lc; + bool first = true; + + /* Take locks if using a CachedPlan */ + myeflags = 0; + if (portal->cplan) + myeflags |= EXEC_FLAG_GET_LOCKS; + + foreach(lc, portal->stmts) + { + PlannedStmt *plan = lfirst_node(PlannedStmt, lc); + bool is_utility = (plan->utilityStmt != NULL); + + /* + * Push the snapshot to be used by the executor. + */ + if (!is_utility) + { + /* + * Must copy the snapshot if we'll need to update + * its command ID. + */ + if (!first) + PushCopiedSnapshot(GetTransactionSnapshot()); + else + PushActiveSnapshot(GetTransactionSnapshot()); + } + + /* + * From the 2nd statement onwards, update the command + * ID and the snapshot to match. + */ + if (!first) + { + CommandCounterIncrement(); + UpdateActiveSnapshotCommandId(); + } + + first = false; + + /* + * Create the QueryDesc object. DestReceiver will + * be set in PortalRunMulti(). + */ + queryDesc = CreateQueryDesc(plan, portal->cplan, + portal->sourceText, + !is_utility ? + GetActiveSnapshot() : + InvalidSnapshot, + InvalidSnapshot, + NULL, + params, + portal->queryEnv, 0); + + /* Remember for PortalRunMulti() */ + portal->qdescs = lappend(portal->qdescs, queryDesc); + + if (is_utility) + continue; + + /* + * Call ExecutorStart to prepare the plan for + * execution. A cached plan may get invalidated as + * we're doing that. + */ + ExecutorStart(queryDesc, myeflags); + PopActiveSnapshot(); + if (!queryDesc->plan_valid) + { + Assert(queryDesc->cplan); + PortalQueryFinish(queryDesc); + portal->plan_valid = false; + goto early_exit; + } + } + } + portal->tupDesc = NULL; + portal->plan_valid = true; break; } } @@ -594,19 +595,18 @@ PortalStart(Portal portal, ParamListInfo params, /* Restore global vars and propagate error */ ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; PG_RE_THROW(); } PG_END_TRY(); + portal->status = PORTAL_READY; + +early_exit: MemoryContextSwitchTo(oldContext); ActivePortal = saveActivePortal; CurrentResourceOwner = saveResourceOwner; - PortalContext = savePortalContext; - - portal->status = PORTAL_READY; } /* @@ -1193,7 +1193,7 @@ PortalRunMulti(Portal portal, QueryCompletion *qc) { bool active_snapshot_set = false; - ListCell *stmtlist_item; + ListCell *qdesc_item; /* * If the destination is DestRemoteExecute, change to DestNone. The @@ -1214,9 +1214,10 @@ PortalRunMulti(Portal portal, * Loop to handle the individual queries generated from a single parsetree * by analysis and rewrite. */ - foreach(stmtlist_item, portal->stmts) + foreach(qdesc_item, portal->qdescs) { - PlannedStmt *pstmt = lfirst_node(PlannedStmt, stmtlist_item); + QueryDesc *qdesc = (QueryDesc *) lfirst(qdesc_item); + PlannedStmt *pstmt = qdesc->plannedstmt; /* * If we got a cancel signal in prior command, quit @@ -1271,23 +1272,38 @@ PortalRunMulti(Portal portal, else UpdateActiveSnapshotCommandId(); + /* + * Run the plan to completion. + */ + qdesc->dest = dest; + ExecutorRun(qdesc, ForwardScanDirection, 0L, true); + + /* + * Build command completion status data if needed. + */ if (pstmt->canSetTag) { - /* statement can set tag string */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - dest, qc); - } - else - { - /* stmt added by rewrite cannot set tag */ - ProcessQuery(pstmt, - portal->sourceText, - portal->portalParams, - portal->queryEnv, - altdest, NULL); + switch (qdesc->operation) + { + case CMD_SELECT: + SetQueryCompletion(qc, CMDTAG_SELECT, qdesc->estate->es_processed); + break; + case CMD_INSERT: + SetQueryCompletion(qc, CMDTAG_INSERT, qdesc->estate->es_processed); + break; + case CMD_UPDATE: + SetQueryCompletion(qc, CMDTAG_UPDATE, qdesc->estate->es_processed); + break; + case CMD_DELETE: + SetQueryCompletion(qc, CMDTAG_DELETE, qdesc->estate->es_processed); + break; + case CMD_MERGE: + SetQueryCompletion(qc, CMDTAG_MERGE, qdesc->estate->es_processed); + break; + default: + SetQueryCompletion(qc, CMDTAG_UNKNOWN, qdesc->estate->es_processed); + break; + } } if (log_executor_stats) @@ -1346,8 +1362,15 @@ PortalRunMulti(Portal portal, * Increment command counter between queries, but not after the last * one. */ - if (lnext(portal->stmts, stmtlist_item) != NULL) + if (lnext(portal->qdescs, qdesc_item) != NULL) CommandCounterIncrement(); + + if (qdesc->estate) + { + ExecutorFinish(qdesc); + ExecutorEnd(qdesc); + } + FreeQueryDesc(qdesc); } /* Pop the snapshot if we pushed one. */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index c7607895cd..014cd476f4 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -2073,6 +2073,27 @@ get_rel_persistence(Oid relid) return result; } +/* + * get_rel_relisshared + * + * Returns if the given relation is shared or not + */ +bool +get_rel_relisshared(Oid relid) +{ + HeapTuple tp; + Form_pg_class reltup; + bool result; + + tp = SearchSysCache1(RELOID, ObjectIdGetDatum(relid)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for relation %u", relid); + reltup = (Form_pg_class) GETSTRUCT(tp); + result = reltup->relisshared; + ReleaseSysCache(tp); + + return result; +} /* ---------- TRANSFORM CACHE ---------- */ diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 77c2ba3f8f..4e455d815f 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,13 +100,13 @@ static void ReleaseGenericPlan(CachedPlanSource *plansource); static List *RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); +static bool GenericPlanIsValid(CachedPlan *cplan); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); static bool choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams); static double cached_plan_cost(CachedPlan *plan, bool include_planner); static Query *QueryListGetPrimaryStmt(List *stmts); -static void AcquireExecutorLocks(List *stmt_list, bool acquire); static void AcquirePlannerLocks(List *stmt_list, bool acquire); static void ScanQueryForLocks(Query *parsetree, bool acquire); static bool ScanQueryWalker(Node *node, bool *acquire); @@ -787,9 +787,6 @@ RevalidateCachedQuery(CachedPlanSource *plansource, * * Caller must have already called RevalidateCachedQuery to verify that the * querytree is up to date. - * - * On a "true" return, we have acquired the locks needed to run the plan. - * (We must do this for the "true" result to be race-condition-free.) */ static bool CheckCachedPlan(CachedPlanSource *plansource) @@ -803,60 +800,56 @@ CheckCachedPlan(CachedPlanSource *plansource) if (!plan) return false; - Assert(plan->magic == CACHEDPLAN_MAGIC); - /* Generic plans are never one-shot */ - Assert(!plan->is_oneshot); + if (GenericPlanIsValid(plan)) + return true; /* - * If plan isn't valid for current role, we can't use it. + * Plan has been invalidated, so unlink it from the parent and release it. */ - if (plan->is_valid && plan->dependsOnRole && - plan->planRoleId != GetUserId()) - plan->is_valid = false; + ReleaseGenericPlan(plansource); - /* - * If it appears valid, acquire locks and recheck; this is much the same - * logic as in RevalidateCachedQuery, but for a plan. - */ - if (plan->is_valid) + return false; +} + +/* + * GenericPlanIsValid + * Is a generic plan still valid? + * + * It may have gone stale due to concurrent schema modifications of relations + * mentioned in the plan or a couple of other things mentioned below. + */ +static bool +GenericPlanIsValid(CachedPlan *cplan) +{ + Assert(cplan != NULL); + Assert(cplan->magic == CACHEDPLAN_MAGIC); + /* Generic plans are never one-shot */ + Assert(!cplan->is_oneshot); + + if (cplan->is_valid) { /* * Plan must have positive refcount because it is referenced by * plansource; so no need to fear it disappears under us here. */ - Assert(plan->refcount > 0); - - AcquireExecutorLocks(plan->stmt_list, true); + Assert(cplan->refcount > 0); /* - * If plan was transient, check to see if TransactionXmin has - * advanced, and if so invalidate it. + * If plan isn't valid for current role, we can't use it. */ - if (plan->is_valid && - TransactionIdIsValid(plan->saved_xmin) && - !TransactionIdEquals(plan->saved_xmin, TransactionXmin)) - plan->is_valid = false; + if (cplan->dependsOnRole && cplan->planRoleId != GetUserId()) + cplan->is_valid = false; /* - * By now, if any invalidation has happened, the inval callback - * functions will have marked the plan invalid. + * If plan was transient, check to see if TransactionXmin has + * advanced, and if so invalidate it. */ - if (plan->is_valid) - { - /* Successfully revalidated and locked the query. */ - return true; - } - - /* Oops, the race case happened. Release useless locks. */ - AcquireExecutorLocks(plan->stmt_list, false); + if (TransactionIdIsValid(cplan->saved_xmin) && + !TransactionIdEquals(cplan->saved_xmin, TransactionXmin)) + cplan->is_valid = false; } - /* - * Plan has been invalidated, so unlink it from the parent and release it. - */ - ReleaseGenericPlan(plansource); - - return false; + return cplan->is_valid; } /* @@ -1126,9 +1119,6 @@ cached_plan_cost(CachedPlan *plan, bool include_planner) * plan or a custom plan for the given parameters: the caller does not know * which it will get. * - * On return, the plan is valid and we have sufficient locks to begin - * execution. - * * On return, the refcount of the plan has been incremented; a later * ReleaseCachedPlan() call is expected. If "owner" is not NULL then * the refcount has been reported to that ResourceOwner (note that this @@ -1360,8 +1350,8 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, } /* - * Reject if AcquireExecutorLocks would have anything to do. This is - * probably unnecessary given the previous check, but let's be safe. + * Reject if the executor would need to take additional locks, that is, in + * addition to those taken by AcquirePlannerLocks() on a given query. */ foreach(lc, plan->stmt_list) { @@ -1735,58 +1725,6 @@ QueryListGetPrimaryStmt(List *stmts) return NULL; } -/* - * AcquireExecutorLocks: acquire locks needed for execution of a cached plan; - * or release them if acquire is false. - */ -static void -AcquireExecutorLocks(List *stmt_list, bool acquire) -{ - ListCell *lc1; - - foreach(lc1, stmt_list) - { - PlannedStmt *plannedstmt = lfirst_node(PlannedStmt, lc1); - ListCell *lc2; - - if (plannedstmt->commandType == CMD_UTILITY) - { - /* - * Ignore utility statements, except those (such as EXPLAIN) that - * contain a parsed-but-not-planned query. Note: it's okay to use - * ScanQueryForLocks, even though the query hasn't been through - * rule rewriting, because rewriting doesn't change the query - * representation. - */ - Query *query = UtilityContainsQuery(plannedstmt->utilityStmt); - - if (query) - ScanQueryForLocks(query, acquire); - continue; - } - - foreach(lc2, plannedstmt->rtable) - { - RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2); - - if (!(rte->rtekind == RTE_RELATION || - (rte->rtekind == RTE_SUBQUERY && OidIsValid(rte->relid)))) - continue; - - /* - * Acquire the appropriate type of lock on each relation OID. Note - * that we don't actually try to open the rel, and hence will not - * fail if it's been dropped entirely --- we'll just transiently - * acquire a non-conflicting lock. - */ - if (acquire) - LockRelationOid(rte->relid, rte->rellockmode); - else - UnlockRelationOid(rte->relid, rte->rellockmode); - } - } -} - /* * AcquirePlannerLocks: acquire locks needed for planning of a querytree list; * or release them if acquire is false. diff --git a/src/backend/utils/mmgr/portalmem.c b/src/backend/utils/mmgr/portalmem.c index 06dfa85f04..3ad80c7ecb 100644 --- a/src/backend/utils/mmgr/portalmem.c +++ b/src/backend/utils/mmgr/portalmem.c @@ -201,6 +201,10 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) portal->portalContext = AllocSetContextCreate(TopPortalContext, "PortalContext", ALLOCSET_SMALL_SIZES); + /* initialize portal's query context to store QueryDescs */ + portal->queryContext = AllocSetContextCreate(TopPortalContext, + "PortalQueryContext", + ALLOCSET_SMALL_SIZES); /* create a resource owner for the portal */ portal->resowner = ResourceOwnerCreate(CurTransactionResourceOwner, @@ -224,6 +228,7 @@ CreatePortal(const char *name, bool allowDup, bool dupSilent) /* for named portals reuse portal->name copy */ MemoryContextSetIdentifier(portal->portalContext, portal->name[0] ? portal->name : "<unnamed>"); + MemoryContextSetIdentifier(portal->queryContext, portal->name[0] ? portal->name : "<unnamed>"); return portal; } @@ -594,6 +599,7 @@ PortalDrop(Portal portal, bool isTopCommit) /* release subsidiary storage */ MemoryContextDelete(portal->portalContext); + MemoryContextDelete(portal->queryContext); /* release portal struct (it's in TopPortalContext) */ pfree(portal); diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h index 3d3e632a0c..392abb5150 100644 --- a/src/include/commands/explain.h +++ b/src/include/commands/explain.h @@ -88,7 +88,11 @@ extern void ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv); -extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, +extern QueryDesc *ExplainQueryDesc(PlannedStmt *stmt, struct CachedPlan *cplan, + const char *queryString, IntoClause *into, ExplainState *es, + ParamListInfo params, QueryEnvironment *queryEnv); +extern void ExplainOnePlan(QueryDesc *queryDesc, + IntoClause *into, ExplainState *es, const char *queryString, ParamListInfo params, QueryEnvironment *queryEnv, const instr_time *planduration, @@ -104,6 +108,7 @@ extern void ExplainQueryParameters(ExplainState *es, ParamListInfo params, int m extern void ExplainBeginOutput(ExplainState *es); extern void ExplainEndOutput(ExplainState *es); +extern void ExplainResetOutput(ExplainState *es); extern void ExplainSeparatePlans(ExplainState *es); extern void ExplainPropertyList(const char *qlabel, List *data, diff --git a/src/include/executor/execdesc.h b/src/include/executor/execdesc.h index af2bf36dfb..c36c25b497 100644 --- a/src/include/executor/execdesc.h +++ b/src/include/executor/execdesc.h @@ -32,9 +32,12 @@ */ typedef struct QueryDesc { + NodeTag type; + /* These fields are provided by CreateQueryDesc */ CmdType operation; /* CMD_SELECT, CMD_UPDATE, etc. */ PlannedStmt *plannedstmt; /* planner's output (could be utility, too) */ + struct CachedPlan *cplan; /* CachedPlan, if plannedstmt is from one */ const char *sourceText; /* source text of the query */ Snapshot snapshot; /* snapshot to use for query */ Snapshot crosscheck_snapshot; /* crosscheck for RI update/delete */ @@ -47,6 +50,7 @@ typedef struct QueryDesc TupleDesc tupDesc; /* descriptor for result tuples */ EState *estate; /* executor's query-wide state */ PlanState *planstate; /* tree of per-plan-node state */ + bool plan_valid; /* is planstate tree fully valid? */ /* This field is set by ExecutorRun */ bool already_executed; /* true if previously executed */ @@ -57,6 +61,7 @@ typedef struct QueryDesc /* in pquery.c */ extern QueryDesc *CreateQueryDesc(PlannedStmt *plannedstmt, + struct CachedPlan *cplan, const char *sourceText, Snapshot snapshot, Snapshot crosscheck_snapshot, diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h index f9e6bf3d4a..a6ac772400 100644 --- a/src/include/executor/executor.h +++ b/src/include/executor/executor.h @@ -19,6 +19,7 @@ #include "nodes/lockoptions.h" #include "nodes/parsenodes.h" #include "utils/memutils.h" +#include "utils/plancache.h" /* @@ -61,6 +62,10 @@ * WITH_NO_DATA indicates that we are performing REFRESH MATERIALIZED VIEW * ... WITH NO DATA. Currently, the only effect is to suppress errors about * scanning unpopulated materialized views. + * + * GET_LOCKS indicates that the caller of ExecutorStart() is executing a + * cached plan which must be validated by taking the remaining locks necessary + * for execution. */ #define EXEC_FLAG_EXPLAIN_ONLY 0x0001 /* EXPLAIN, no ANALYZE */ #define EXEC_FLAG_EXPLAIN_GENERIC 0x0002 /* EXPLAIN (GENERIC_PLAN) */ @@ -69,6 +74,8 @@ #define EXEC_FLAG_MARK 0x0010 /* need mark/restore */ #define EXEC_FLAG_SKIP_TRIGGERS 0x0020 /* skip AfterTrigger setup */ #define EXEC_FLAG_WITH_NO_DATA 0x0040 /* REFRESH ... WITH NO DATA */ +#define EXEC_FLAG_GET_LOCKS 0x0400 /* should the executor lock + * relations? */ /* Hook for plugins to get control in ExecutorStart() */ @@ -255,6 +262,13 @@ extern void ExecEndNode(PlanState *node); extern void ExecShutdownNode(PlanState *node); extern void ExecSetTupleBound(int64 tuples_needed, PlanState *child_node); +/* Is the cached plan*/ +static inline bool +ExecPlanStillValid(EState *estate) +{ + return estate->es_cachedplan == NULL ? true : + CachedPlanStillValid(estate->es_cachedplan); +} /* ---------------------------------------------------------------- * ExecProcNode @@ -589,6 +603,8 @@ exec_rt_fetch(Index rti, EState *estate) } extern Relation ExecGetRangeTableRelation(EState *estate, Index rti); +extern void ExecLockViewRelations(List *viewRelations, EState *estate); +extern void ExecLockAppendNonLeafRelations(EState *estate, List *allpartrelids); extern void ExecInitResultRelation(EState *estate, ResultRelInfo *resultRelInfo, Index rti); diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index d97f5a8e7d..dfa72848c7 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -623,6 +623,8 @@ typedef struct EState * ExecRowMarks, or NULL if none */ List *es_rteperminfos; /* List of RTEPermissionInfo */ PlannedStmt *es_plannedstmt; /* link to top of plan tree */ + struct CachedPlan *es_cachedplan; /* CachedPlan if plannedstmt is from + * one */ List *es_part_prune_infos; /* PlannedStmt.partPruneInfos */ const char *es_sourceText; /* Source text from QueryDesc */ diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h index d61a62da19..9b888b0d75 100644 --- a/src/include/nodes/pathnodes.h +++ b/src/include/nodes/pathnodes.h @@ -116,6 +116,9 @@ typedef struct PlannerGlobal /* "flat" list of RTEPermissionInfos */ List *finalrteperminfos; + /* "flat" list of integer RT indexes */ + List *viewRelations; + /* "flat" list of PlanRowMarks */ List *finalrowmarks; diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index a0bb16cff4..7cae624bbd 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes/plannodes.h @@ -78,6 +78,9 @@ typedef struct PlannedStmt List *permInfos; /* list of RTEPermissionInfo nodes for rtable * entries needing one */ + List *viewRelations; /* integer list of RT indexes, or NIL if no + * views are queried */ + /* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */ List *resultRelations; /* integer list of RT indexes, or NIL */ diff --git a/src/include/storage/lmgr.h b/src/include/storage/lmgr.h index 4ee91e3cf9..598bf2688a 100644 --- a/src/include/storage/lmgr.h +++ b/src/include/storage/lmgr.h @@ -48,6 +48,7 @@ extern bool ConditionalLockRelation(Relation relation, LOCKMODE lockmode); extern void UnlockRelation(Relation relation, LOCKMODE lockmode); extern bool CheckRelationLockedByMe(Relation relation, LOCKMODE lockmode, bool orstronger); +extern bool CheckRelLockedByMe(Oid relid, LOCKMODE lockmode, bool orstronger); extern bool LockHasWaitersRelation(Relation relation, LOCKMODE lockmode); extern void LockRelationIdForSession(LockRelId *relid, LOCKMODE lockmode); diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 4f5418b972..3074e604dd 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -139,6 +139,7 @@ extern char get_rel_relkind(Oid relid); extern bool get_rel_relispartition(Oid relid); extern Oid get_rel_tablespace(Oid relid); extern char get_rel_persistence(Oid relid); +extern bool get_rel_relisshared(Oid relid); extern Oid get_transform_fromsql(Oid typid, Oid langid, List *trftypes); extern Oid get_transform_tosql(Oid typid, Oid langid, List *trftypes); extern bool get_typisdefined(Oid typid); diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a443181d41..8990fe72e3 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -221,6 +221,20 @@ extern CachedPlan *GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams, ResourceOwner owner, QueryEnvironment *queryEnv); + +/* + * CachedPlanStillValid + * Returns if a cached generic plan is still valid + * + * Called by the executor on every relation lock taken when initializing the + * plan tree in the CachedPlan. + */ +static inline bool +CachedPlanStillValid(CachedPlan *cplan) +{ + return cplan->is_valid; +} + extern void ReleaseCachedPlan(CachedPlan *plan, ResourceOwner owner); extern bool CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource, diff --git a/src/include/utils/portal.h b/src/include/utils/portal.h index aa08b1e0fc..332a08ccb4 100644 --- a/src/include/utils/portal.h +++ b/src/include/utils/portal.h @@ -138,6 +138,9 @@ typedef struct PortalData QueryCompletion qc; /* command completion data for executed query */ List *stmts; /* list of PlannedStmts */ CachedPlan *cplan; /* CachedPlan, if stmts are from one */ + List *qdescs; /* list of QueryDescs */ + bool plan_valid; /* are plan(s) ready for execution? */ + MemoryContext queryContext; /* memory for QueryDescs and children */ ParamListInfo portalParams; /* params to pass to query */ QueryEnvironment *queryEnv; /* environment for query */ @@ -242,6 +245,7 @@ extern void PortalDefineQuery(Portal portal, CommandTag commandTag, List *stmts, CachedPlan *cplan); +extern void PortalQueryFinish(QueryDesc *queryDesc); extern PlannedStmt *PortalGetPrimaryStmt(Portal portal); extern void PortalCreateHoldStore(Portal portal); extern void PortalHashTableDeleteAll(void); diff --git a/src/test/modules/delay_execution/Makefile b/src/test/modules/delay_execution/Makefile index 70f24e846d..2fca84d027 100644 --- a/src/test/modules/delay_execution/Makefile +++ b/src/test/modules/delay_execution/Makefile @@ -8,7 +8,8 @@ OBJS = \ delay_execution.o ISOLATION = partition-addition \ - partition-removal-1 + partition-removal-1 \ + cached-plan-replan ifdef USE_PGXS PG_CONFIG = pg_config diff --git a/src/test/modules/delay_execution/delay_execution.c b/src/test/modules/delay_execution/delay_execution.c index 7cd76eb34b..5d7a3e9858 100644 --- a/src/test/modules/delay_execution/delay_execution.c +++ b/src/test/modules/delay_execution/delay_execution.c @@ -1,14 +1,18 @@ /*------------------------------------------------------------------------- * * delay_execution.c - * Test module to allow delay between parsing and execution of a query. + * Test module to introduce delay at various points during execution of a + * query to test that execution proceeds safely in light of concurrent + * changes. * * The delay is implemented by taking and immediately releasing a specified * advisory lock. If another process has previously taken that lock, the * current process will be blocked until the lock is released; otherwise, * there's no effect. This allows an isolationtester script to reliably - * test behaviors where some specified action happens in another backend - * between parsing and execution of any desired query. + * test behaviors where some specified action happens in another backend in + * a couple of cases: 1) between parsing and execution of any desired query + * when using the planner_hook, 2) between RevalidateCachedQuery() and + * ExecutorStart() when using the ExecutorStart_hook. * * Copyright (c) 2020-2023, PostgreSQL Global Development Group * @@ -22,6 +26,7 @@ #include <limits.h> +#include "executor/executor.h" #include "optimizer/planner.h" #include "utils/builtins.h" #include "utils/guc.h" @@ -32,9 +37,11 @@ PG_MODULE_MAGIC; /* GUC: advisory lock ID to use. Zero disables the feature. */ static int post_planning_lock_id = 0; +static int executor_start_lock_id = 0; -/* Save previous planner hook user to be a good citizen */ +/* Save previous hook users to be a good citizen */ static planner_hook_type prev_planner_hook = NULL; +static ExecutorStart_hook_type prev_ExecutorStart_hook = NULL; /* planner_hook function to provide the desired delay */ @@ -70,11 +77,41 @@ delay_execution_planner(Query *parse, const char *query_string, return result; } +/* planner_hook function to provide the desired delay */ +static void +delay_execution_ExecutorStart(QueryDesc *queryDesc, int eflags) +{ + /* If enabled, delay by taking and releasing the specified lock */ + if (executor_start_lock_id != 0) + { + DirectFunctionCall1(pg_advisory_lock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + DirectFunctionCall1(pg_advisory_unlock_int8, + Int64GetDatum((int64) executor_start_lock_id)); + + /* + * Ensure that we notice any pending invalidations, since the advisory + * lock functions don't do this. + */ + AcceptInvalidationMessages(); + } + + /* Now start the executor, possibly via a previous hook user */ + if (prev_ExecutorStart_hook) + prev_ExecutorStart_hook(queryDesc, eflags); + else + standard_ExecutorStart(queryDesc, eflags); + + if (executor_start_lock_id != 0) + elog(NOTICE, "Finished ExecutorStart(): CachedPlan is %s", + queryDesc->cplan->is_valid ? "valid" : "not valid"); +} + /* Module load function */ void _PG_init(void) { - /* Set up the GUC to control which lock is used */ + /* Set up GUCs to control which lock is used */ DefineCustomIntVariable("delay_execution.post_planning_lock_id", "Sets the advisory lock ID to be locked/unlocked after planning.", "Zero disables the delay.", @@ -86,10 +123,22 @@ _PG_init(void) NULL, NULL, NULL); - + DefineCustomIntVariable("delay_execution.executor_start_lock_id", + "Sets the advisory lock ID to be locked/unlocked before starting execution.", + "Zero disables the delay.", + &executor_start_lock_id, + 0, + 0, INT_MAX, + PGC_USERSET, + 0, + NULL, + NULL, + NULL); MarkGUCPrefixReserved("delay_execution"); - /* Install our hook */ + /* Install our hooks. */ prev_planner_hook = planner_hook; planner_hook = delay_execution_planner; + prev_ExecutorStart_hook = ExecutorStart_hook; + ExecutorStart_hook = delay_execution_ExecutorStart; } diff --git a/src/test/modules/delay_execution/expected/cached-plan-replan.out b/src/test/modules/delay_execution/expected/cached-plan-replan.out new file mode 100644 index 0000000000..4f450b9d9b --- /dev/null +++ b/src/test/modules/delay_execution/expected/cached-plan-replan.out @@ -0,0 +1,117 @@ +Parsed test spec with 2 sessions + +starting permutation: s1prep s2lock s1exec s2dropi s2unlock +step s1prep: SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); +QUERY PLAN +-------------------------------------------- +Append + Subplans Removed: 1 + -> Bitmap Heap Scan on foo11 foo_1 + Recheck Cond: (a = $1) + -> Bitmap Index Scan on foo11_a_idx + Index Cond: (a = $1) +(6 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +----------------------------- +Append + Subplans Removed: 1 + -> Seq Scan on foo11 foo_1 + Filter: (a = $1) +(4 rows) + + +starting permutation: s1prep2 s2lock s1exec2 s2dropi s2unlock +step s1prep2: SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +---------------------------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Index Only Scan using foo11_a_idx on foo11 t1 + -> Materialize + -> Index Scan using foo11_a_idx on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(18 rows) + +step s2lock: SELECT pg_advisory_lock(12345); +pg_advisory_lock +---------------- + +(1 row) + +step s1exec2: LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; <waiting ...> +step s2dropi: DROP INDEX foo11_a; +step s2unlock: SELECT pg_advisory_unlock(12345); +pg_advisory_unlock +------------------ +t +(1 row) + +step s1exec2: <... completed> +s1: NOTICE: Finished ExecutorStart(): CachedPlan is not valid +s1: NOTICE: Finished ExecutorStart(): CachedPlan is valid +QUERY PLAN +--------------------------------------------- +Append + -> GroupAggregate + Group Key: t1.a + -> Merge Join + Merge Cond: (t1.a = t2.a) + -> Sort + Sort Key: t1.a + -> Seq Scan on foo11 t1 + -> Sort + Sort Key: t2.a + -> Seq Scan on foo11 t2 + -> GroupAggregate + Group Key: t1_1.a + -> Merge Join + Merge Cond: (t1_1.a = t2_1.a) + -> Sort + Sort Key: t1_1.a + -> Seq Scan on foo2 t1_1 + -> Sort + Sort Key: t2_1.a + -> Seq Scan on foo2 t2_1 +(21 rows) + diff --git a/src/test/modules/delay_execution/specs/cached-plan-replan.spec b/src/test/modules/delay_execution/specs/cached-plan-replan.spec new file mode 100644 index 0000000000..67cfed7044 --- /dev/null +++ b/src/test/modules/delay_execution/specs/cached-plan-replan.spec @@ -0,0 +1,50 @@ +# Test to check that invalidation of a cached plan during ExecutorStart +# correctly triggers replanning and re-execution. + +setup +{ + CREATE TABLE foo (a int, b text) PARTITION BY LIST(a); + CREATE TABLE foo1 PARTITION OF foo FOR VALUES IN (1) PARTITION BY LIST (a); + CREATE TABLE foo11 PARTITION OF foo1 FOR VALUES IN (1); + CREATE INDEX foo11_a ON foo1 (a); + CREATE TABLE foo2 PARTITION OF foo FOR VALUES IN (2); + CREATE VIEW foov AS SELECT * FROM foo; +} + +teardown +{ + DROP VIEW foov; + DROP TABLE foo; +} + +session "s1" +# Creates a prepared statement and forces creation of a generic plan +step "s1prep" { SET plan_cache_mode = force_generic_plan; + PREPARE q AS SELECT * FROM foov WHERE a = $1; + EXPLAIN (COSTS OFF) EXECUTE q (1); } + +step "s1prep2" { SET plan_cache_mode = force_generic_plan; + SET enable_partitionwise_aggregate = on; + SET enable_partitionwise_join = on; + PREPARE q2 AS SELECT t1.a, count(t2.b) FROM foo t1, foo t2 WHERE t1.a = t2.a GROUP BY 1; + EXPLAIN (COSTS OFF) EXECUTE q2; } +# Executes a generic plan +step "s1exec" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q (1); } +step "s1exec2" { LOAD 'delay_execution'; + SET delay_execution.executor_start_lock_id = 12345; + EXPLAIN (COSTS OFF) EXECUTE q2; } + +session "s2" +step "s2lock" { SELECT pg_advisory_lock(12345); } +step "s2unlock" { SELECT pg_advisory_unlock(12345); } +step "s2dropi" { DROP INDEX foo11_a; } + +# While "s1exec" waits to acquire the advisory lock, "s2drop" is able to drop +# the index being used in the cached plan for `q`, so when "s1exec" is then +# unblocked and initializes the cached plan for execution, it detects the +# concurrent index drop and causes the cached plan to be discarded and +# recreated without the index. +permutation "s1prep" "s2lock" "s1exec" "s2dropi" "s2unlock" +permutation "s1prep2" "s2lock" "s1exec2" "s2dropi" "s2unlock" -- 2.35.3 ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-04-03 21:41 Tom Lane <[email protected]> parent: Amit Langote <[email protected]> 0 siblings, 1 reply; 22+ messages in thread From: Tom Lane @ 2023-04-03 21:41 UTC (permalink / raw) To: Amit Langote <[email protected]>; +Cc: Andres Freund <[email protected]>; Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]> Amit Langote <[email protected]> writes: > [ v38 patchset ] I spent a little bit of time looking through this, and concluded that it's not something I will be wanting to push into v16 at this stage. The patch doesn't seem very close to being committable on its own terms, and even if it was now is not a great time in the dev cycle to be making significant executor API changes. Too much risk of having to thrash the API during beta, or even change it some more in v17. I suggest that we push this forward to the next CF with the hope of landing it early in v17. A few concrete thoughts: * I understand that your plan now is to acquire locks on all the originally-named tables, then do permissions checks (which will involve only those tables), then dynamically lock just inheritance and partitioning child tables as we descend the plan tree. That seems more or less okay to me, but it could be reflected better in the structure of the patch perhaps. * In particular I don't much like the "viewRelations" list, which seems like a wart; those ought to be handled more nearly the same way as other RTEs. (One concrete reason why is that this scheme is going to result in locking views in a different order than they were locked during original parsing, which perhaps could contribute to deadlocks.) Maybe we should store an integer list of which RTIs need to be locked in the early phase? Building that in the parser/rewriter would provide a solid guide to the original locking order, so we'd be trivially sure of duplicating that. (It might be close enough to follow the RT list order, which is basically what AcquireExecutorLocks does today, but this'd be more certain to do the right thing.) I'm less concerned about lock order for child tables because those are just going to follow the inheritance or partitioning structure. * I don't understand the need for changes like this: /* clean up tuple table */ - ExecClearTuple(node->ps.ps_ResultTupleSlot); + if (node->ps.ps_ResultTupleSlot) + ExecClearTuple(node->ps.ps_ResultTupleSlot); ISTM that the process ought to involve taking a lock (if needed) before we have built any execution state for a given plan node, and if we find we have to fail, returning NULL instead of a partially-valid planstate node. Otherwise, considerations of how to handle partially-valid nodes are going to metastasize into all sorts of places, almost certainly including EXPLAIN for instance. I think we ought to be able to limit the damage to "parent nodes might have NULL child links that you wouldn't have expected". That wouldn't faze ExecEndNode at all, nor most other code. * More attention is needed to comments. For example, in a couple of places in plancache.c you have removed function header comments defining API details and not replaced them with any info about the new details, despite the fact that those details are more complex than the old. > It seems I hadn't noted in the ExecEndNode()'s comment that all node > types' recursive subroutines need to handle the change made by this > patch that the corresponding ExecInitNode() subroutine may now return > early without having initialized all state struct fields. > Also noted in the documentation for CustomScan and ForeignScan that > the Begin*Scan callback may not have been called at all, so the > End*Scan should handle that gracefully. Yeah, I think we need to avoid adding such requirements. It's the sort of thing that would far too easily get past developer testing and only fail once in a blue moon in the field. regards, tom lane ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2023-04-04 13:29 Amit Langote <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: Amit Langote @ 2023-04-04 13:29 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Alvaro Herrera <[email protected]>; David Rowley <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers; Robert Haas <[email protected]> On Tue, Apr 4, 2023 at 6:41 AM Tom Lane <[email protected]> wrote: > Amit Langote <[email protected]> writes: > > [ v38 patchset ] > > I spent a little bit of time looking through this, and concluded that > it's not something I will be wanting to push into v16 at this stage. > The patch doesn't seem very close to being committable on its own > terms, and even if it was now is not a great time in the dev cycle > to be making significant executor API changes. Too much risk of > having to thrash the API during beta, or even change it some more > in v17. I suggest that we push this forward to the next CF with the > hope of landing it early in v17. OK, thanks a lot for your feedback. > A few concrete thoughts: > > * I understand that your plan now is to acquire locks on all the > originally-named tables, then do permissions checks (which will > involve only those tables), then dynamically lock just inheritance and > partitioning child tables as we descend the plan tree. Actually, with the current implementation of the patch, *all* of the relations mentioned in the plan tree would get locked during the ExecInitNode() traversal of the plan tree (and of those in plannedstmt->subplans), not just the inheritance child tables. Locking of non-child tables done by the executor after this patch is duplicative with AcquirePlannerLocks(), so that's something to be improved. > That seems > more or less okay to me, but it could be reflected better in the > structure of the patch perhaps. > > * In particular I don't much like the "viewRelations" list, which > seems like a wart; those ought to be handled more nearly the same way > as other RTEs. (One concrete reason why is that this scheme is going > to result in locking views in a different order than they were locked > during original parsing, which perhaps could contribute to deadlocks.) > Maybe we should store an integer list of which RTIs need to be locked > in the early phase? Building that in the parser/rewriter would provide > a solid guide to the original locking order, so we'd be trivially sure > of duplicating that. (It might be close enough to follow the RT list > order, which is basically what AcquireExecutorLocks does today, but > this'd be more certain to do the right thing.) I'm less concerned > about lock order for child tables because those are just going to > follow the inheritance or partitioning structure. What you've described here sounds somewhat like what I had implemented in the patch versions till v31, though it used a bitmapset named minLockRelids that is initialized by setrefs.c. Your idea of initializing a list before planning seems more appealing offhand than the code I had added in setrefs.c to populate that minLockRelids bitmapset, which would be bms_add_range(1, list_lenth(finalrtable)), followed by bms_del_members(set-of-child-rel-rtis). I'll give your idea a try. > * I don't understand the need for changes like this: > > /* clean up tuple table */ > - ExecClearTuple(node->ps.ps_ResultTupleSlot); > + if (node->ps.ps_ResultTupleSlot) > + ExecClearTuple(node->ps.ps_ResultTupleSlot); > > ISTM that the process ought to involve taking a lock (if needed) > before we have built any execution state for a given plan node, > and if we find we have to fail, returning NULL instead of a > partially-valid planstate node. Otherwise, considerations of how > to handle partially-valid nodes are going to metastasize into all > sorts of places, almost certainly including EXPLAIN for instance. > I think we ought to be able to limit the damage to "parent nodes > might have NULL child links that you wouldn't have expected". > That wouldn't faze ExecEndNode at all, nor most other code. Hmm, yes, taking a lock before allocating any of the stuff to add into the planstate seems like it's much easier to reason about than the alternative I've implemented. > * More attention is needed to comments. For example, in a couple of > places in plancache.c you have removed function header comments > defining API details and not replaced them with any info about the new > details, despite the fact that those details are more complex than the > old. OK, yeah, maybe I've added a bunch of explanations in execMain.c that should perhaps have been in plancache.c. > > It seems I hadn't noted in the ExecEndNode()'s comment that all node > > types' recursive subroutines need to handle the change made by this > > patch that the corresponding ExecInitNode() subroutine may now return > > early without having initialized all state struct fields. > > Also noted in the documentation for CustomScan and ForeignScan that > > the Begin*Scan callback may not have been called at all, so the > > End*Scan should handle that gracefully. > > Yeah, I think we need to avoid adding such requirements. It's the > sort of thing that would far too easily get past developer testing > and only fail once in a blue moon in the field. OK, got it. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2024-05-19 00:39 David Rowley <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 2 replies; 22+ messages in thread From: David Rowley @ 2024-05-19 00:39 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Amit Langote <[email protected]>; Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers On Fri, 20 Jan 2023 at 08:39, Tom Lane <[email protected]> wrote: > I spent some time re-reading this whole thread, and the more I read > the less happy I got. We are adding a lot of complexity and introducing > coding hazards that will surely bite somebody someday. And after awhile > I had what felt like an epiphany: the whole problem arises because the > system is wrongly factored. We should get rid of AcquireExecutorLocks > altogether, allowing the plancache to hand back a generic plan that > it's not certain of the validity of, and instead integrate the > responsibility for acquiring locks into executor startup. It'd have > to be optional there, since we don't need new locks in the case of > executing a just-planned plan; but we can easily add another eflags > bit (EXEC_FLAG_GET_LOCKS or so). Then there has to be a convention > whereby the ExecInitNode traversal can return an indicator that > "we failed because the plan is stale, please make a new plan". I also reread the entire thread up to this point yesterday. I've also been thinking about this recently as Amit has mentioned it to me a few times over the past few months. With the caveat of not yet having looked at the latest patch, my thoughts are that having the executor startup responsible for taking locks is a bad idea and I don't think we should go down this path. My reasons are: 1. No ability to control the order that the locks are obtained. The order in which the locks are taken will be at the mercy of the plan the planner chooses. 2. It introduces lots of complexity regarding how to cleanly clean up after a failed executor startup which is likely to make exec startup slower and the code more complex 3. It puts us even further down the path of actually needing an executor startup phase. For #1, the locks taken for SELECT queries are less likely to conflict with other locks obtained by PostgreSQL, but at least at the moment if someone is getting deadlocks with a DDL type operation, they can change their query or DDL script so that locks are taken in the same order. If we allowed executor startup to do this then if someone comes complaining that PG18 deadlocks when PG17 didn't we'd just have to tell them to live with it. There's a comment at the bottom of find_inheritance_children_extended() just above the qsort() which explains about the deadlocking issue. I don't have much extra to say about #2. As mentioned, I've not looked at the patch. On paper, it sounds possible, but it also sounds bug-prone and ugly. For #3, I've been thinking about what improvements we can do to make the executor more efficient. In [1], Andres talks about some very interesting things. In particular, in his email items 3) and 5) are relevant here. If we did move lots of executor startup code into the planner, I think it would be possible to one day get rid of executor startup and have the plan record how much memory is needed for the non-readonly part of the executor state and tag each plan node with the offset in bytes they should use for their portion of the executor working state. This would be a single memory allocation for the entire plan. The exact details are not important here, but I feel like if we load up executor startup with more responsibilities, it'll just make doing something like this harder. The init run-time pruning code that I worked on likely already has done that, but I don't think it's closed the door on it as it might just mean allocating more executor state memory than we need to. Providing the plan node records the offset into that memory, I think it could be made to work, just with the inefficiency of having a (possibly) large unused hole in that state memory. As far as I understand it, your objection to the original proposal is just on the grounds of concerns about introducing hazards that could turn into bugs. I think we could come up with some way to make the prior method of doing pruning before executor startup work. I think what Amit had before your objection was starting to turn into something workable and we should switch back to working on that. David [1] https://www.postgresql.org/message-id/20180525033538.6ypfwcqcxce6zkjj%40alap3.anarazel.de ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2024-05-19 01:27 Tom Lane <[email protected]> parent: David Rowley <[email protected]> 1 sibling, 1 reply; 22+ messages in thread From: Tom Lane @ 2024-05-19 01:27 UTC (permalink / raw) To: David Rowley <[email protected]>; +Cc: Amit Langote <[email protected]>; Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers David Rowley <[email protected]> writes: > With the caveat of not yet having looked at the latest patch, my > thoughts are that having the executor startup responsible for taking > locks is a bad idea and I don't think we should go down this path. OK, it's certainly still up for argument, but ... > 1. No ability to control the order that the locks are obtained. The > order in which the locks are taken will be at the mercy of the plan > the planner chooses. I do not think I buy this argument, because plancache.c doesn't provide any "ability to control the order" today, and never has. The order in which AcquireExecutorLocks re-gets relation locks is only weakly related to the order in which the parser/planner got them originally. The order in which AcquirePlannerLocks re-gets the locks is even less related to the original. This doesn't cause any big problems that I'm aware of, because these locks are fairly weak. I think we do have a guarantee that for partitioned tables, parents will be locked before children, and that's probably valuable. But an executor-driven lock order could preserve that property too. > 2. It introduces lots of complexity regarding how to cleanly clean up > after a failed executor startup which is likely to make exec startup > slower and the code more complex Perhaps true, I'm not sure. But the patch we'd been discussing before this proposal was darn complex as well. > 3. It puts us even further down the path of actually needing an > executor startup phase. Huh? We have such a thing already. > For #1, the locks taken for SELECT queries are less likely to conflict > with other locks obtained by PostgreSQL, but at least at the moment if > someone is getting deadlocks with a DDL type operation, they can > change their query or DDL script so that locks are taken in the same > order. If we allowed executor startup to do this then if someone > comes complaining that PG18 deadlocks when PG17 didn't we'd just have > to tell them to live with it. There's a comment at the bottom of > find_inheritance_children_extended() just above the qsort() which > explains about the deadlocking issue. The reason it's important there is that function is (sometimes) used for lock modes that *are* exclusive. > For #3, I've been thinking about what improvements we can do to make > the executor more efficient. In [1], Andres talks about some very > interesting things. In particular, in his email items 3) and 5) are > relevant here. If we did move lots of executor startup code into the > planner, I think it would be possible to one day get rid of executor > startup and have the plan record how much memory is needed for the > non-readonly part of the executor state and tag each plan node with > the offset in bytes they should use for their portion of the executor > working state. I'm fairly skeptical about that idea. The entire reason we have an issue here is that we want to do runtime partition pruning, which by definition can't be done at plan time. So I doubt it's going to play nice with what we are trying to accomplish in this thread. Moreover, while "replace a bunch of small pallocs with one big one" would save some palloc effort, what are you going to do to ensure that that memory has the right initial contents? I think this idea is likely to make the executor a great deal more notationally complex without actually buying all that much. Maybe Andres can make it work, but I don't want to contort other parts of the system design on the purely hypothetical basis that this might happen. > I think what Amit had before your objection was starting to turn into > something workable and we should switch back to working on that. The reason I posted this idea was that I didn't think the previously existing patch looked promising at all. regards, tom lane ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2024-05-19 01:51 David Rowley <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 22+ messages in thread From: David Rowley @ 2024-05-19 01:51 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Amit Langote <[email protected]>; Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers On Sun, 19 May 2024 at 13:27, Tom Lane <[email protected]> wrote: > > David Rowley <[email protected]> writes: > > 1. No ability to control the order that the locks are obtained. The > > order in which the locks are taken will be at the mercy of the plan > > the planner chooses. > > I do not think I buy this argument, because plancache.c doesn't > provide any "ability to control the order" today, and never has. > The order in which AcquireExecutorLocks re-gets relation locks is only > weakly related to the order in which the parser/planner got them > originally. The order in which AcquirePlannerLocks re-gets the locks > is even less related to the original. This doesn't cause any big > problems that I'm aware of, because these locks are fairly weak. It may not bite many people, it's just that if it does, I don't see what we could do to help those people. At the moment we could tell them to adjust their DDL script to obtain the locks in the same order as their query. With your idea that cannot be done as the order could change when the planner switches the join order. > I think we do have a guarantee that for partitioned tables, parents > will be locked before children, and that's probably valuable. > But an executor-driven lock order could preserve that property too. I think you'd have to lock the parent before the child. That would remain true and consistent anyway when taking locks during a breadth-first plan traversal. > > For #3, I've been thinking about what improvements we can do to make > > the executor more efficient. In [1], Andres talks about some very > > interesting things. In particular, in his email items 3) and 5) are > > relevant here. If we did move lots of executor startup code into the > > planner, I think it would be possible to one day get rid of executor > > startup and have the plan record how much memory is needed for the > > non-readonly part of the executor state and tag each plan node with > > the offset in bytes they should use for their portion of the executor > > working state. > > I'm fairly skeptical about that idea. The entire reason we have an > issue here is that we want to do runtime partition pruning, which > by definition can't be done at plan time. So I doubt it's going > to play nice with what we are trying to accomplish in this thread. I think we could have both, providing there was a way to still traverse the executor state tree in EXPLAIN. We'd need a way to skip portions of the plan that are not relevant or could be invalid for the current execution. e.g can't show Index Scan because index has been dropped. > > I think what Amit had before your objection was starting to turn into > > something workable and we should switch back to working on that. > > The reason I posted this idea was that I didn't think the previously > existing patch looked promising at all. Ok. It would be good if you could expand on that so we could determine if there's some fundamental reason it can't work or if that's because you were blinded by your epiphany and didn't give that any thought after thinking of the alternative idea. I've gone to effort to point out things that I think are concerning with your idea. It would be good if you could do the same for the previous patch other than "it didn't look promising". It's pretty hard for me to argue with that level of detail. David ^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: generic plans and "initial" pruning @ 2024-05-20 12:13 Amit Langote <[email protected]> parent: David Rowley <[email protected]> 1 sibling, 0 replies; 22+ messages in thread From: Amit Langote @ 2024-05-20 12:13 UTC (permalink / raw) To: David Rowley <[email protected]>; +Cc: Tom Lane <[email protected]>; Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Jacob Champion <[email protected]>; pgsql-hackers On Sun, May 19, 2024 at 9:39 AM David Rowley <[email protected]> wrote: > For #1, the locks taken for SELECT queries are less likely to conflict > with other locks obtained by PostgreSQL, but at least at the moment if > someone is getting deadlocks with a DDL type operation, they can > change their query or DDL script so that locks are taken in the same > order. If we allowed executor startup to do this then if someone > comes complaining that PG18 deadlocks when PG17 didn't we'd just have > to tell them to live with it. There's a comment at the bottom of > find_inheritance_children_extended() just above the qsort() which > explains about the deadlocking issue. Thought to chime in on this. A deadlock may occur with the execution-time locking proposed in the patch if the DDL script makes assumptions about how a cached plan's execution determines the locking order for children of multiple parent relations. Specifically, the deadlock can happen if the script tries to lock the child relations directly, instead of locking them through their respective parent relations. The patch doesn't change the order of locking of relations mentioned in the query, because that's defined in AcquirePlannerLocks(). -- Thanks, Amit Langote ^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2024-05-20 12:13 UTC | newest] Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-01-19 19:39 Re: generic plans and "initial" pruning Tom Lane <[email protected]> 2023-01-20 03:13 ` Amit Langote <[email protected]> 2023-01-20 03:31 ` Tom Lane <[email protected]> 2023-01-20 03:52 ` Amit Langote <[email protected]> 2023-01-20 03:58 ` Tom Lane <[email protected]> 2023-01-20 07:19 ` Amit Langote <[email protected]> 2023-01-27 07:01 ` Amit Langote <[email protected]> 2023-02-02 14:49 ` Amit Langote <[email protected]> 2023-02-03 13:01 ` Amit Langote <[email protected]> 2023-02-07 18:08 ` Andres Freund <[email protected]> 2023-02-08 10:31 ` Amit Langote <[email protected]> 2023-03-02 13:52 ` Amit Langote <[email protected]> 2023-03-14 10:07 ` Amit Langote <[email protected]> 2023-03-22 12:48 ` Amit Langote <[email protected]> 2023-03-27 08:18 ` Amit Langote <[email protected]> 2023-03-27 14:00 ` Amit Langote <[email protected]> 2023-04-03 21:41 ` Tom Lane <[email protected]> 2023-04-04 13:29 ` Amit Langote <[email protected]> 2024-05-19 00:39 ` David Rowley <[email protected]> 2024-05-19 01:27 ` Tom Lane <[email protected]> 2024-05-19 01:51 ` David Rowley <[email protected]> 2024-05-20 12:13 ` Amit Langote <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox