public inbox for [email protected]
help / color / mirror / Atom feedFrom: Lukas Fittl <[email protected]>
To: Sami Imseih <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Marko M <[email protected]>
Subject: Re: [PATCH] Optionally record Plan IDs to track plan changes for a query
Date: Tue, 4 Feb 2025 18:09:50 -0800
Message-ID: <CAP53PkxocbNr+eRag3FEJp3-7S1U80FspOg8UQjO902TWMG=6A@mail.gmail.com> (raw)
In-Reply-To: <CAA5RZ0sfRbd1xcq_oHNA0TPr57yM8qkg-GoD6A0nUfyxZhs33Q@mail.gmail.com>
References: <CAP53Pkyow59ajFMHGpmb1BK9WHDypaWtUsS_5DoYUEfsa_Hktg@mail.gmail.com>
<[email protected]>
<CAA5RZ0sUPPOpkRZD=Za83op2ngcPC7dp249vcHA-X5YS7p3n8Q@mail.gmail.com>
<CAP53PkwuFbo3NkwZgxwNRMjMfqPEqidD-SggaoQ4ijotBVLJAA@mail.gmail.com>
<[email protected]>
<[email protected]>
<CAP53Pkw1QJHH9UDjkArS=XdAxtK7XWMpZLGHnVMDmhRTp_HYYw@mail.gmail.com>
<[email protected]>
<CAA5RZ0vM9AsEqvKued2drKZJ1opt3wbYaDbxGzi-khkNzwn7og@mail.gmail.com>
<[email protected]>
<CAA5RZ0sfRbd1xcq_oHNA0TPr57yM8qkg-GoD6A0nUfyxZhs33Q@mail.gmail.com>
Looks like some emails were sent before I could send my draft email, but
hopefully this should make the follow up work easier :)
Attached a v4 patch set with a few minor changes to plan ID jumbling:
* Range table jumbling is now done in a separate JumbleRangeTable function
after setrefs.c walked the tree - this way we avoid having custom logic for
RT Indexes in the node jumbling, and keeping a reference to PlannerGlobal
in the jumble struct
* Moved the JumbleNode call to the bottom of the set_plan_references
function for clarity - previously it was before descending into inner/outer
plan, but after some other recursive calls to set_plan_references, which
didn't really make sense
* Fixed a bug with JUMBLE_ARRAY incorrectly taking the reference of the
array (which caused planid to change incorrectly between runs)
* Added JUMBLE_BITMAPSET
Further, I've significantly reduced the number of fields ignored for plan
jumbling:
Basically the approach taken in this version is that only things that would
negatively affect the planid (i.e. make it unique when it shouldn't be) are
ignored, vs ignoring duplicate fields and fields that are only used by the
executor (which is what v1-v3 did). I'm not 100% sure that's the right
approach (but it does keep the diff a good amount smaller), I think the
tradeoff here is basically jumbling performance vs maintenance overhead
when fields are added/changed.
This does not yet move field-specific comments to their own line in nodes
where we're adding node attributes, I'll leave that for Sami to work on.
On Tue, Feb 4, 2025 at 3:15 PM Sami Imseih <[email protected]> wrote:
> >> Separately I've been thinking how we could best have a
> discussion/review on
> >> whether the jumbling of specific plan struct fields is correct. I was
> >> thinking maybe a quick wiki page could be helpful, noting why to
> jumble/not
> >> jumble certain fields?
>
> > Makes sense. This is a complicated topic.
>
> +1 for the Wiki page
>
Started here: https://wiki.postgresql.org/wiki/Plan_ID_Jumbling
Thanks,
Lukas
--
Lukas Fittl
Attachments:
[application/octet-stream] v4-0002-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patch (54.6K, 3-v4-0002-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patch)
download | inline diff:
From 59227e046cf2848b34ac77741d040b6e489be1bd Mon Sep 17 00:00:00 2001
From: Lukas Fittl <[email protected]>
Date: Sun, 2 Feb 2025 12:18:49 -0800
Subject: [PATCH v4 2/3] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
doc/src/sgml/config.sgml | 34 +++
doc/src/sgml/monitoring.sgml | 16 ++
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 16 ++
src/backend/executor/execMain.c | 10 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/gen_node_support.pl | 36 ++-
src/backend/nodes/queryjumblefuncs.c | 65 +++++
src/backend/optimizer/plan/planner.c | 19 ++
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 243 ++++++++++++------
src/include/nodes/primnodes.h | 7 +-
src/include/nodes/queryjumble.h | 30 ++-
src/include/utils/backend_status.h | 5 +
src/test/regress/expected/explain.out | 11 +
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
25 files changed, 529 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f109982..85a1e0d8d4b 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8406,6 +8406,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index edc2470bcf9..64e62a09059 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index cddc3ea9b53..d3d5578d2c8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -889,6 +889,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e1..31a1761f539 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 604cb0625b8..6890082d745 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -121,13 +121,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id / plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9c313d81315..da2279579a8 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 1a657f7e0ae..6c20bfc5986 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1281,13 +1281,18 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_location = 0;
# extract per-field attributes
foreach my $a (@a)
{
- if ($a eq 'query_jumble_ignore')
+ if ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1297,8 +1302,15 @@ _jumble${n}(JumbleState *jstate, Node *node)
}
}
+ next if $query_jumble_ignore;
+
+ if ($t eq 'Bitmapset*')
+ {
+ print $jff "\tJUMBLE_BITMAPSET($f);\n"
+ unless $query_jumble_ignore;
+ }
# node type
- if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
print $jff "\tJUMBLE_NODE($f);\n"
@@ -1318,6 +1330,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 545d8edcae2..64bec54a211 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -35,12 +35,14 @@
#include "common/hashfn.h"
#include "miscadmin.h"
#include "nodes/queryjumble.h"
+#include "parser/parsetree.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -51,6 +53,15 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or plan_query_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void RecordConstLocation(JumbleState *jstate, int location);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
@@ -171,6 +182,19 @@ EnableQueryId(void)
query_id_enabled = true;
}
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
/*
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
@@ -240,6 +264,13 @@ RecordConstLocation(JumbleState *jstate, int location)
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_BITMAPSET(item) \
+do { \
+ if (expr->item) \
+ AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
+} while(0)
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len)
#define JUMBLE_FIELD_SINGLE(item) \
AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
#define JUMBLE_STRING(str) \
@@ -388,3 +419,37 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_FIELD(is_local);
JUMBLE_LOCATION(location);
}
+
+/*
+ * Jumble the entries in the rangle table to map RT indexes to relations
+ *
+ * This ensures jumbled RT indexes (e.g. in a Scan or Modify node), are
+ * distinguished by the target of the RT entry, even if the index is the same.
+ */
+void
+JumbleRangeTable(JumbleState *jstate, List *rtable)
+{
+ ListCell *lc;
+
+ foreach(lc, rtable)
+ {
+ RangeTblEntry *expr = lfirst_node(RangeTblEntry, lc);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+ }
+}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ffd7517ea97..9d43a174b9e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -532,6 +533,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -596,6 +607,14 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->jitFlags |= PGJIT_DEFORM;
}
+ if (IsPlanIdEnabled())
+ {
+ JumbleRangeTable(glob->plan_jumble_state, glob->finalrtable);
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
if (glob->partition_directory != NULL)
DestroyPartitionDirectory(glob->partition_directory);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0868249be94..5e890f576b0 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1306,6 +1307,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
return plan;
}
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6da..654acf5bf04 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e2..6d8947bae9b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a6..1dfb7a58f87 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as
+ * it'll only be known after parse analysis / planning, to avoid reporting
+ * last query's identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0, it
+ * means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e9096a88492..a3d5592beb6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -330,7 +330,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -641,6 +641,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -670,6 +674,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 71448bb4fdd..22fdfb60ccc 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4873,6 +4891,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa7..0634ae90dd3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b8c2ad2a54..5547db7d2f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5568,9 +5568,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 52d44f43021..4baf829f644 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -166,6 +166,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier calculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 06d9559ebb9..2ccf772d87e 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,10 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (calculated if
+ * compute_plan_id is enabled, can also be set
+ * by plugins) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -121,44 +125,55 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ int disabled_nodes pg_node_attr(query_jumble_ignore); /* count of disabled
+ * nodes */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* cost expended before
+ * fetching any tuples */
+ Cost total_cost pg_node_attr(query_jumble_ignore); /* total cost (assuming
+ * all tuples fetched) */
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore); /* number of rows plan
+ * is expected to emit */
+ int plan_width pg_node_attr(query_jumble_ignore); /* average row width in
+ * bytes */
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_aware; /* engage parallel-aware
+ * logic? */
+ bool parallel_safe; /* OK to use as part of
+ * parallel plan? */
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ bool async_capable; /* engage
+ * asynchronous-capable
+ * logic? */
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
+ int plan_node_id; /* unique across entire
+ * final plan tree */
List *targetlist; /* target list to be computed at this node */
List *qual; /* implicitly-ANDed qual conditions */
- struct Plan *lefttree; /* input plan tree(s) */
- struct Plan *righttree;
- List *initPlan; /* Init Plan nodes (un-correlated expr
- * subselects) */
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore); /* input plan tree(s) */
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
+ List *initPlan pg_node_attr(query_jumble_ignore); /* Init Plan nodes
+ * (un-correlated expr
+ * subselects) */
/*
* Information for management of parameter-change-driven rescanning
@@ -234,31 +249,47 @@ typedef struct ModifyTable
{
Plan plan;
CmdType operation; /* INSERT, UPDATE, DELETE, or MERGE */
- bool canSetTag; /* do we set the command tag/es_processed? */
- Index nominalRelation; /* Parent RT index for use of EXPLAIN */
- Index rootRelation; /* Root RT index, if partitioned/inherited */
+ bool canSetTag; /* do we set the command
+ * tag/es_processed? */
+ Index nominalRelation; /* Parent RT index for
+ * use of EXPLAIN */
+ Index rootRelation; /* Root RT index, if
+ * partitioned/inherited */
bool partColsUpdated; /* some part key in hierarchy updated? */
- List *resultRelations; /* integer list of RT indexes */
- List *updateColnosLists; /* per-target-table update_colnos lists */
- List *withCheckOptionLists; /* per-target-table WCO lists */
- char *returningOldAlias; /* alias for OLD in RETURNING lists */
- char *returningNewAlias; /* alias for NEW in RETURNING lists */
- List *returningLists; /* per-target-table RETURNING tlists */
- List *fdwPrivLists; /* per-target-table FDW private data lists */
- Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
- List *rowMarks; /* PlanRowMarks (non-locking only) */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *resultRelations; /* integer list of RT
+ * indexes */
+ List *updateColnosLists; /* per-target-table
+ * update_colnos lists */
+ List *withCheckOptionLists; /* per-target-table WCO
+ * lists */
+ char *returningOldAlias; /* alias for OLD in
+ * RETURNING lists */
+ char *returningNewAlias; /* alias for NEW in
+ * RETURNING lists */
+ List *returningLists; /* per-target-table
+ * RETURNING tlists */
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore); /* per-target-table FDW
+ * private data lists */
+ Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM
+ * plans */
+ List *rowMarks; /* PlanRowMarks
+ * (non-locking only) */
+ int epqParam; /* ID of Param for
+ * EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* INSERT ON CONFLICT DO UPDATE targetlist */
List *onConflictCols; /* target column numbers for onConflictSet */
Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
- Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
- List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
- List *mergeActionLists; /* per-target-table lists of actions for
- * MERGE */
- List *mergeJoinConditions; /* per-target-table join conditions
- * for MERGE */
+ Index exclRelRTI; /* RTI of the EXCLUDED
+ * pseudo relation */
+ List *exclRelTlist; /* tlist of the EXCLUDED
+ * pseudo relation */
+ List *mergeActionLists; /* per-target-table
+ * lists of actions for
+ * MERGE */
+ List *mergeJoinConditions; /* per-target-table join
+ * conditions for MERGE */
} ModifyTable;
struct PartitionPruneInfo; /* forward reference to struct below */
@@ -272,7 +303,7 @@ typedef struct Append
{
Plan plan;
Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
- List *appendplans;
+ List *appendplans pg_node_attr(query_jumble_ignore);
int nasyncplans; /* # of asynchronous plans */
/*
@@ -301,7 +332,7 @@ typedef struct MergeAppend
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
@@ -356,7 +387,7 @@ typedef struct RecursiveUnion
Oid *dupCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -370,7 +401,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -385,7 +416,7 @@ typedef struct BitmapOr
{
Plan plan;
bool isshared;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -400,7 +431,8 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ Index scanrelid; /* relid is index into
+ * the range table */
} Scan;
/* ----------------
@@ -465,9 +497,11 @@ typedef struct IndexScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig; /* the same in original
+ * form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indexorderbyorig; /* the same in original form */
+ List *indexorderbyorig; /* the same in original
+ * form */
List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexScan;
@@ -508,9 +542,12 @@ typedef struct IndexOnlyScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *recheckqual; /* index quals in recheckable form */
+ List *recheckqual; /* index quals in
+ * recheckable form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indextlist; /* TargetEntry list describing index's cols */
+ List *indextlist; /* TargetEntry list
+ * describing index's
+ * cols */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexOnlyScan;
@@ -535,9 +572,11 @@ typedef struct BitmapIndexScan
{
Scan scan;
Oid indexid; /* OID of index to scan */
- bool isshared; /* Create shared bitmap if set */
+ bool isshared; /* Create shared bitmap
+ * if set */
List *indexqual; /* list of index quals (OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig; /* the same in original
+ * form */
} BitmapIndexScan;
/* ----------------
@@ -552,7 +591,8 @@ typedef struct BitmapIndexScan
typedef struct BitmapHeapScan
{
Scan scan;
- List *bitmapqualorig; /* index quals, in standard expr form */
+ List *bitmapqualorig; /* index quals, in
+ * standard expr form */
} BitmapHeapScan;
/* ----------------
@@ -612,7 +652,7 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
SubqueryScanStatus scanstatus;
} SubqueryScan;
@@ -654,8 +694,11 @@ typedef struct TableFuncScan
typedef struct CteScan
{
Scan scan;
- int ctePlanId; /* ID of init SubPlan for CTE */
- int cteParam; /* ID of Param representing CTE output */
+ int ctePlanId; /* ID of init SubPlan
+ * for CTE */
+ int cteParam; /* ID of Param
+ * representing CTE
+ * output */
} CteScan;
/* ----------------
@@ -675,7 +718,9 @@ typedef struct NamedTuplestoreScan
typedef struct WorkTableScan
{
Scan scan;
- int wtParam; /* ID of Param representing work table */
+ int wtParam; /* ID of Param
+ * representing work
+ * table */
} WorkTableScan;
/* ----------------
@@ -722,17 +767,26 @@ typedef struct ForeignScan
{
Scan scan;
CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
- Index resultRelation; /* direct modification target's RT index */
- Oid checkAsUser; /* user to perform the scan as; 0 means to
- * check as current user */
+ Index resultRelation; /* direct modification
+ * target's RT index */
+ Oid checkAsUser; /* user to perform the
+ * scan as; 0 means to
+ * check as current user */
Oid fs_server; /* OID of foreign server */
- List *fdw_exprs; /* expressions that FDW may evaluate */
- List *fdw_private; /* private data for FDW */
- List *fdw_scan_tlist; /* optional tlist describing scan tuple */
- List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
- bool fsSystemCol; /* true if any "system column" is needed */
+ List *fdw_exprs; /* expressions that FDW
+ * may evaluate */
+ List *fdw_private pg_node_attr(query_jumble_ignore); /* private data for FDW */
+ List *fdw_scan_tlist; /* optional tlist
+ * describing scan tuple */
+ List *fdw_recheck_quals; /* original quals not in
+ * scan.plan.qual */
+ Bitmapset *fs_relids; /* base+OJ RTIs
+ * generated by this
+ * scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated
+ * by this scan */
+ bool fsSystemCol; /* true if any "system
+ * column" is needed */
} ForeignScan;
/* ----------------
@@ -753,20 +807,27 @@ struct CustomScanMethods;
typedef struct CustomScan
{
Scan scan;
- uint32 flags; /* mask of CUSTOMPATH_* flags, see
- * nodes/extensible.h */
- List *custom_plans; /* list of Plan nodes, if any */
- List *custom_exprs; /* expressions that custom code may evaluate */
- List *custom_private; /* private data for custom code */
- List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ uint32 flags; /* mask of CUSTOMPATH_*
+ * flags, see
+ * nodes/extensible.h */
+ List *custom_plans; /* list of Plan nodes,
+ * if any */
+ List *custom_exprs; /* expressions that
+ * custom code may
+ * evaluate */
+ List *custom_private pg_node_attr(query_jumble_ignore); /* private data for
+ * custom code */
+ List *custom_scan_tlist; /* optional tlist
+ * describing scan tuple */
+ Bitmapset *custom_relids; /* RTIs generated by
+ * this scan */
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -826,7 +887,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
int paramno; /* number of the PARAM_EXEC Param to set */
@@ -932,7 +993,7 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
Bitmapset *keyparamids;
@@ -1028,7 +1089,7 @@ typedef struct Agg
Oid *grpCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
uint64 transitionSpace;
@@ -1154,12 +1215,19 @@ typedef struct Unique
typedef struct Gather
{
Plan plan;
- int num_workers; /* planned number of worker processes */
- int rescan_param; /* ID of Param that signals a rescan, or -1 */
- bool single_copy; /* don't execute plan more than once */
- bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
- * at gather or one of its child nodes */
+ /* planned number of worker processes */
+ int num_workers;
+ /* ID of Param that signals a rescan, or -1 */
+ int rescan_param;
+ /* don't execute plan more than once */
+ bool single_copy;
+ /* suppress EXPLAIN display (for testing)? */
+ bool invisible;
+ /*
+ * param id's of initplans which are referred at gather or one of its
+ * child nodes
+ */
+ Bitmapset *initParam;
} Gather;
/* ------------
@@ -1217,11 +1285,16 @@ typedef struct Hash
* needed to put them into the hashtable.
*/
List *hashkeys; /* hash keys for the hashjoin condition */
- Oid skewTable; /* outer join key's table OID, or InvalidOid */
- AttrNumber skewColumn; /* outer join key's column #, or zero */
- bool skewInherit; /* is outer join rel an inheritance tree? */
+ Oid skewTable; /* outer join key's
+ * table OID, or
+ * InvalidOid */
+ AttrNumber skewColumn; /* outer join key's
+ * column #, or zero */
+ bool skewInherit; /* is outer join rel an
+ * inheritance tree? */
/* all other info is in the parent HashJoin node */
- Cardinality rows_total; /* estimate total rows if parallel_aware */
+ Cardinality rows_total pg_node_attr(query_jumble_ignore); /* estimate total rows
+ * if parallel_aware */
} Hash;
/* ----------------
@@ -1252,7 +1325,7 @@ typedef struct SetOp
bool *cmpNullsFirst pg_node_attr(array_size(numCols));
/* estimated number of groups in left input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
@@ -1267,8 +1340,10 @@ typedef struct SetOp
typedef struct LockRows
{
Plan plan;
- List *rowMarks; /* a list of PlanRowMark's */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *rowMarks; /* a list of
+ * PlanRowMark's */
+ int epqParam; /* ID of Param for
+ * EvalPlanQual re-eval */
} LockRows;
/* ----------------
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 839e71d52f4..85702f75070 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1075,8 +1075,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1107,8 +1105,9 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan
+ * evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 5afa6f3605f..6356dd73fbc 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -59,15 +60,27 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,10 +96,25 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
-/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
extern JumbleState *InitializeJumbleState(bool record_clocations);
extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
extern void JumbleNode(JumbleState *jstate, Node *node);
+extern void JumbleRangeTable(JumbleState *jstate, List *rtable);
extern uint64 HashJumbleState(JumbleState *jstate);
#endif /* QUERYJUMBLE_H */
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c9..437a4cec5bf 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d508..8bfa3c1a5f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3361f6a69c9..6778b79e658 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1890,7 +1891,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2096,7 +2097,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2130,7 +2131,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa870496..d787ad2cda3 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
--
2.47.1
[application/octet-stream] v4-0003-Add-pg_stat_plans-contrib-extension.patch (71.0K, 4-v4-0003-Add-pg_stat_plans-contrib-extension.patch)
download | inline diff:
From 2c39ebd625e20784daa0e15541d5a417142139e7 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Fri, 31 Jan 2025 13:08:19 +0900
Subject: [PATCH v4 3/3] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 33 +
contrib/pg_stat_plans/pg_stat_plans.c | 779 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 1 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 425 ++++++++++
17 files changed, 1834 insertions(+)
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
create mode 100644 doc/src/sgml/pgstatplans.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b61..8de010afdef 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67a..0442ec2644a 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 00000000000..e073db95edd
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 00000000000..3e21d6d7019
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 00000000000..906d8ce90d6
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 00000000000..3bd884d9601
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 00000000000..f08452b274b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,33 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT toplevel bool,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 00000000000..5fa31fcbf3e
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,779 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/queryjumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in
+ * bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report
+ * stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every
+ * pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see
+ * their own query plans */
+ bool toplevel; /* query executed at top level */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to
+ * prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /*
+ * We currently don't write to a file since plan texts would get lost (and
+ * just the stats on their own aren't that useful)
+ */
+ .write_to_file = false,
+
+ /*
+ * Plan statistics are available system-wide to simplify monitoring
+ * scripts
+ */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id, toplevel) hash_combine64(toplevel, hash_combine64(query_id, hash_combine64(plan_id, user_id)))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different
+ * costs, and it could be misleading to only show the first recorded
+ * plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /*
+ * Clean up this entry's plan text allocation, if we haven't done so
+ * already
+ */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Encourage other backends to clean up dropped entry refs */
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+ bool toplevel = (nesting_level == 0);
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid, toplevel), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry (we do
+ * this whilst holding the lock on the new entry so we don't remove it
+ * by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.toplevel = toplevel;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /*
+ * Increase refcount here so entry can't get released without us
+ * dropping the plan text
+ */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions evaluated during
+ * planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /*
+ * Record initial entry now, so plan text is available for currently
+ * running queries
+ */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in
+ * pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */ );
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the module
+ * isn't active. The functions must protect themselves against being
+ * called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto, as well as plan_id calculation if
+ * compute_plan_id is set to auto.
+ */
+ EnableQueryId();
+ EnablePlanId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+static bool
+match_plans_entries(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_PLANS;
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_matching_entries(match_plans_entries, 0);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 8
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_get_entry_data(p->key.kind, dsa_get_address(pgStatLocal.dsa, p->body));
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ values[i++] = BoolGetDatum(statent->info.toplevel);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 00000000000..6750b3e2cc0
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1 @@
+shared_preload_libraries = 'pg_stat_plans'
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 00000000000..4db3a47239b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 00000000000..aaad72a6553
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 00000000000..f0e803ad70c
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a53..4a5a02c7049 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c9..b0afb33ce22 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 00000000000..2d5faff15a9
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,425 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to
+ <literal>auto</literal> or <literal>on</literal> and
+ <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, whether it's a top-level statement or not, query ID and plan ID
+ (up to the maximum number of distinct plans that the module can track).
+ The columns of the view are shown in <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>toplevel</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the query was executed as a top-level statement
+ (always true if <varname>pg_stat_plans.track</varname> is set to
+ <literal>top</literal>)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ will have the same <structfield>planid</structfield> whenever they have identical plan
+ structures according to an internal hash calculation. Typically, two plans will be
+ considered the same for this purpose if they have the same
+ <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically equivalent except
+ for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>,
+ <structfield>userid</structfield> and <structfield>toplevel</structfield>
+ as a stable and reliable identifier for each entry, instead of using its
+ plan text. However, it is important to understand that there are only limited
+ guarantees around the stability of the <structfield>planid</structfield>
+ hash value. Since the identifier is derived from the plan tree, its value
+ is a function of, among other things, the internal object identifiers
+ appearing in this representation. This has some counterintuitive implications.
+ For example, <filename>pg_stat_plans</filename> will consider two
+ apparently-identical plans to be distinct, if they reference a table
+ that was dropped and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>[email protected]</email>.
+ </para>
+ </sect2>
+
+</sect1>
--
2.47.1
[application/octet-stream] v4-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patch (4.9K, 5-v4-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patch)
download | inline diff:
From e590cd2be54923aa608d0e575f272fde84b7a193 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <[email protected]>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH v4 1/3] Allow using jumbling logic outside of query jumble
unit file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
src/backend/nodes/queryjumblefuncs.c | 59 +++++++++++++++++-----------
src/include/nodes/queryjumble.h | 6 +++
2 files changed, 41 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index b103a281936..545d8edcae2 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -51,10 +51,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +106,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +175,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +216,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +235,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -239,8 +250,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -305,7 +316,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 50eb9566587..5afa6f3605f 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
--
2.47.1
view thread (34+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: [PATCH] Optionally record Plan IDs to track plan changes for a query
In-Reply-To: <CAP53PkxocbNr+eRag3FEJp3-7S1U80FspOg8UQjO902TWMG=6A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox