pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Drouvot, Bertrand <bdrouvot@amazon.com>
To: Imseih (AWS), Sami <simseih@amazon.com>
To: Schneider (AWS), Jeremy <schnjere@amazon.com>
To: Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: Pavel Stehule <pavel.stehule@gmail.com>
Cc: Nikolay Samokhvalov <samokhvalov@gmail.com>
Subject: Re: [PATCH] Query Jumbling for CALL and SET utility statements
Date: Fri, 2 Sep 2022 11:06:50 +0200
Message-ID: <09c04aba-7f12-e524-20f7-3f92c1479fdf@amazon.com> (raw )
In-Reply-To: <BBB9A13D-BBE2-44A4-9CA7-6E189120E5A2@amazon.com >
References: <36e5bffe-e989-194f-85c8-06e7bc88e6f7@amazon.com >
<20220831160839.wseeotr7rckucpmv@awork3.anarazel.de >
<46ca04f9-1e37-c862-9a28-c5dffaee412f@amazon.com >
<20220831190604.l2luuhj3xkau7dvk@awork3.anarazel.de >
<e43a0e02-4ef4-d510-a00a-cf39d2a9df19@amazon.com >
<cbd28368-a5ae-f141-4c3f-28ba9f608ba0@amazon.com >
<BBB9A13D-BBE2-44A4-9CA7-6E189120E5A2@amazon.com >
Hi,
On 9/1/22 5:13 PM, Imseih (AWS), Sami wrote:
>
> > Please find attached v2 as an attempt to do so.
>
> +1 to the idea.
>
Thanks for looking at it!
> I think it will be better to evaluate jstate instead of
>
> JUMBLE_UTILITY, such as:
>
> if (query->utilityStmt && !jstate)
>
> instead of
>
> if (query->utilityStmt && !JUMBLE_UTILITY(query->utilityStmt))
>
> This will allow for support of potentially other utility statements
>
> In the future without having to teach pg_stat_statements about them.
>
> If a jstate is set for the utility statements, pgss will do the right
> thing.
>
Fair point, thanks!
v3 including this change is attached.
Thanks,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services:https://aws.amazon.com
Attachments:
v3-0001-JumbleQuery-on-Call-and-Set.patch (text/plain, 15.5K) download
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index ff0166fb9d..e1834b0782 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -272,7 +272,7 @@ FROM pg_stat_statements ORDER BY query COLLATE "C";
wal_records > $2 as wal_records_generated, +| | | | |
wal_records >= rows as wal_records_ge_rows +| | | | |
FROM pg_stat_statements ORDER BY query COLLATE "C" | | | | |
- SET pg_stat_statements.track_utility = FALSE | 1 | 0 | f | f | t
+ SET pg_stat_statements.track_utility = $1 | 1 | 0 | f | f | t
UPDATE pgss_test SET b = $1 WHERE a > $2 | 1 | 3 | t | t | t
(7 rows)
@@ -423,6 +423,147 @@ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
(6 rows)
+-- PL/pgSQL procedure and pg_stat_statements.track = top
+CREATE PROCEDURE MINUS_TWO(i INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (i - 1 - 1.0)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+CREATE PROCEDURE SUM_TWO(i INTEGER, j INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (j + j)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+SET pg_stat_statements.track = 'top';
+SET pg_stat_statements.track_utility = TRUE;
+SELECT pg_stat_statements_reset();
+ pg_stat_statements_reset
+--------------------------
+
+(1 row)
+
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls | rows
+------------------------------------------------------------------------------+-------+------
+ CALL MINUS_TWO($1) | 2 | 0
+ CALL SUM_TWO($1, $2) | 2 | 0
+ SELECT pg_stat_statements_reset() | 1 | 1
+ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
+(4 rows)
+
+-- Ensure CALL is tracked even if pg_stat_statements.track_utility is FALSE
+SET pg_stat_statements.track_utility = FALSE;
+SELECT pg_stat_statements_reset();
+ pg_stat_statements_reset
+--------------------------
+
+(1 row)
+
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls | rows
+------------------------------------------------------------------------------+-------+------
+ CALL MINUS_TWO($1) | 2 | 0
+ CALL SUM_TWO($1, $2) | 2 | 0
+ SELECT pg_stat_statements_reset() | 1 | 1
+ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
+(4 rows)
+
+SET pg_stat_statements.track_utility = TRUE;
+-- PL/pgSQL procedure and pg_stat_statements.track = all
+-- we drop and recreate the procedures to avoid any caching funnies
+SET pg_stat_statements.track_utility = FALSE;
+DROP PROCEDURE MINUS_TWO(INTEGER);
+DROP PROCEDURE SUM_TWO(INTEGER, INTEGER);
+CREATE PROCEDURE MINUS_TWO(i INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (i - 1 - 1.0)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+CREATE PROCEDURE SUM_TWO(i INTEGER, j INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (j + j)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+SET pg_stat_statements.track = 'all';
+SET pg_stat_statements.track_utility = TRUE;
+SELECT pg_stat_statements_reset();
+ pg_stat_statements_reset
+--------------------------
+
+(1 row)
+
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls | rows
+------------------------------------------------------------------------------+-------+------
+ CALL MINUS_TWO($1) | 2 | 0
+ CALL SUM_TWO($1, $2) | 2 | 0
+ SELECT (i - $2 - $3)::INTEGER | 2 | 2
+ SELECT (j + j)::INTEGER | 2 | 2
+ SELECT pg_stat_statements_reset() | 1 | 1
+ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
+(6 rows)
+
+-- Ensure CALL is tracked even if pg_stat_statements.track_utility is FALSE
+SET pg_stat_statements.track_utility = FALSE;
+SELECT pg_stat_statements_reset();
+ pg_stat_statements_reset
+--------------------------
+
+(1 row)
+
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls | rows
+------------------------------------------------------------------------------+-------+------
+ CALL MINUS_TWO($1) | 2 | 0
+ CALL SUM_TWO($1, $2) | 2 | 0
+ SELECT (i - 1 - 1.0)::INTEGER | 2 | 2
+ SELECT (j + j)::INTEGER | 2 | 2
+ SELECT pg_stat_statements_reset() | 1 | 1
+ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
+(6 rows)
+
+SET pg_stat_statements.track_utility = TRUE;
+-- SET
+SELECT pg_stat_statements_reset();
+ pg_stat_statements_reset
+--------------------------
+
+(1 row)
+
+set enable_seqscan=false;
+set enable_seqscan=true;
+set seq_page_cost=2.0;
+set seq_page_cost=1.0;
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+ query | calls | rows
+------------------------------------------------------------------------------+-------+------
+ SELECT pg_stat_statements_reset() | 1 | 1
+ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C" | 0 | 0
+ set enable_seqscan=$1 | 2 | 0
+ set seq_page_cost=$1 | 2 | 0
+(4 rows)
+
+SET pg_stat_statements.track_utility = FALSE;
--
-- queries with locking clauses
--
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index ba868f0de9..62de9b9325 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -106,6 +106,11 @@ static const uint32 PGSS_PG_MAJOR_VERSION = PG_VERSION_NUM / 100;
#define PGSS_HANDLED_UTILITY(n) (!IsA(n, ExecuteStmt) && \
!IsA(n, PrepareStmt) && \
!IsA(n, DeallocateStmt))
+/*
+ * Force track those utility statements
+ * whatever the value of pgss_track_utility is.
+ */
+#define FORCE_TRACK_UTILITY(n) (IsA(n, CallStmt))
/*
* Extension version number, for supporting older extension versions' objects
@@ -832,7 +837,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query, JumbleState *jstate)
* inherit from the underlying statement's one (except DEALLOCATE which is
* entirely untracked).
*/
- if (query->utilityStmt)
+ if (query->utilityStmt && !jstate)
{
if (pgss_track_utility && !PGSS_HANDLED_UTILITY(query->utilityStmt))
query->queryId = UINT64CONST(0);
@@ -1097,7 +1102,8 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* that user configured another extension to handle utility statements
* only.
*/
- if (pgss_enabled(exec_nested_level) && pgss_track_utility)
+ if (pgss_enabled(exec_nested_level) &&
+ (pgss_track_utility || FORCE_TRACK_UTILITY(parsetree)))
pstmt->queryId = UINT64CONST(0);
/*
@@ -1114,7 +1120,8 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
*
* Likewise, we don't track execution of DEALLOCATE.
*/
- if (pgss_track_utility && pgss_enabled(exec_nested_level) &&
+ if ((pgss_track_utility || FORCE_TRACK_UTILITY(parsetree)) &&
+ pgss_enabled(exec_nested_level) &&
PGSS_HANDLED_UTILITY(parsetree))
{
instr_time start;
diff --git a/contrib/pg_stat_statements/sql/pg_stat_statements.sql b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
index a01f183727..92359f0184 100644
--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql
+++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
@@ -201,6 +201,92 @@ SELECT PLUS_ONE(1);
SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+-- PL/pgSQL procedure and pg_stat_statements.track = top
+CREATE PROCEDURE MINUS_TWO(i INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (i - 1 - 1.0)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+
+CREATE PROCEDURE SUM_TWO(i INTEGER, j INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (j + j)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+
+SET pg_stat_statements.track = 'top';
+SET pg_stat_statements.track_utility = TRUE;
+SELECT pg_stat_statements_reset();
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- Ensure CALL is tracked even if pg_stat_statements.track_utility is FALSE
+SET pg_stat_statements.track_utility = FALSE;
+SELECT pg_stat_statements_reset();
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+SET pg_stat_statements.track_utility = TRUE;
+
+-- PL/pgSQL procedure and pg_stat_statements.track = all
+-- we drop and recreate the procedures to avoid any caching funnies
+SET pg_stat_statements.track_utility = FALSE;
+DROP PROCEDURE MINUS_TWO(INTEGER);
+DROP PROCEDURE SUM_TWO(INTEGER, INTEGER);
+
+CREATE PROCEDURE MINUS_TWO(i INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (i - 1 - 1.0)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+
+CREATE PROCEDURE SUM_TWO(i INTEGER, j INTEGER) AS $$
+DECLARE
+ r INTEGER;
+BEGIN
+ SELECT (j + j)::INTEGER INTO r;
+END; $$ LANGUAGE plpgsql;
+
+SET pg_stat_statements.track = 'all';
+SET pg_stat_statements.track_utility = TRUE;
+SELECT pg_stat_statements_reset();
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- Ensure CALL is tracked even if pg_stat_statements.track_utility is FALSE
+SET pg_stat_statements.track_utility = FALSE;
+SELECT pg_stat_statements_reset();
+CALL MINUS_TWO(3);
+CALL MINUS_TWO(7);
+CALL SUM_TWO(3, 8);
+CALL SUM_TWO(7, 5);
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+SET pg_stat_statements.track_utility = TRUE;
+
+-- SET
+SELECT pg_stat_statements_reset();
+set enable_seqscan=false;
+set enable_seqscan=true;
+set seq_page_cost=2.0;
+set seq_page_cost=1.0;
+
+SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+SET pg_stat_statements.track_utility = FALSE;
+
--
-- queries with locking clauses
--
diff --git a/doc/src/sgml/pgstatstatements.sgml b/doc/src/sgml/pgstatstatements.sgml
index ecf6cd6bf3..b1f407e794 100644
--- a/doc/src/sgml/pgstatstatements.sgml
+++ b/doc/src/sgml/pgstatstatements.sgml
@@ -492,7 +492,7 @@
structures according to an internal hash calculation. Typically, two
queries will be considered the same for this purpose if they are
semantically equivalent except for the values of literal constants
- appearing in the query. Utility commands (that is, all other commands)
+ appearing in the query. Utility commands (that is, all other commands) except CALL and SET
are compared strictly on the basis of their textual query strings, however.
</para>
@@ -781,9 +781,10 @@
<listitem>
<para>
<varname>pg_stat_statements.track_utility</varname> controls whether
- utility commands are tracked by the module. Utility commands are
+ most utility commands are tracked by the module. Utility commands are
all those other than <command>SELECT</command>, <command>INSERT</command>,
- <command>UPDATE</command> and <command>DELETE</command>.
+ <command>UPDATE</command> and <command>DELETE</command>, but this parameter does not disable
+ tracking of PREPARE, EXECUTE or CALL.
The default value is <literal>on</literal>.
Only superusers can change this setting.
</para>
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index a67487e5fe..11a57d7616 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -104,7 +104,7 @@ JumbleQuery(Query *query, const char *querytext)
Assert(IsQueryIdEnabled());
- if (query->utilityStmt)
+ if (query->utilityStmt && !JUMBLE_UTILITY(query->utilityStmt))
{
query->queryId = compute_utility_query_id(querytext,
query->stmt_location,
@@ -240,10 +240,11 @@ static void
JumbleQueryInternal(JumbleState *jstate, Query *query)
{
Assert(IsA(query, Query));
- Assert(query->utilityStmt == NULL);
+ Assert(query->utilityStmt == NULL || JUMBLE_UTILITY(query->utilityStmt));
APP_JUMB(query->commandType);
/* resultRelation is usually predictable from commandType */
+ JumbleExpr(jstate, (Node *) query->utilityStmt);
JumbleExpr(jstate, (Node *) query->cteList);
JumbleRangeTable(jstate, query->rtable);
JumbleExpr(jstate, (Node *) query->jointree);
@@ -383,6 +384,30 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varlevelsup);
}
break;
+ case T_CallStmt:
+ {
+ CallStmt *stmt = (CallStmt *) node;
+ FuncExpr *expr = stmt->funcexpr;
+
+ APP_JUMB(expr->funcid);
+ JumbleExpr(jstate, (Node *) expr->args);
+ }
+ break;
+ case T_VariableSetStmt:
+ {
+ VariableSetStmt *stmt = (VariableSetStmt *) node;
+
+ APP_JUMB_STRING(stmt->name);
+ JumbleExpr(jstate, (Node *) stmt->args);
+ }
+ break;
+ case T_A_Const:
+ {
+ int loc = ((const A_Const *) node)->location;
+
+ RecordConstLocation(jstate, loc);
+ }
+ break;
case T_Const:
{
Const *c = (Const *) node;
diff --git a/src/include/utils/queryjumble.h b/src/include/utils/queryjumble.h
index 3c2d9beab2..cada9a1ed3 100644
--- a/src/include/utils/queryjumble.h
+++ b/src/include/utils/queryjumble.h
@@ -18,6 +18,12 @@
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
+/*
+ * Jumble those utility statements
+ */
+#define JUMBLE_UTILITY(n) (IsA(n, CallStmt) || \
+ IsA(n, VariableSetStmt))
+
/*
* Struct for tracking locations/lengths of constants during normalization
*/
view thread (45+ messages) latest in thread
Message-ID: <09c04aba-7f12-e524-20f7-3f92c1479fdf@amazon.com>
Permalink: ../09c04aba-7f12-e524-20f7-3f92c1479fdf@amazon.com/
Also on: postgresql.org/message-id/09c04aba-7f12-e524-20f7-3f92c1479fdf@amazon.com
copy link · copy postgr.es · copy git am
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bdrouvot@amazon.com, simseih@amazon.com, schnjere@amazon.com, andres@anarazel.de, pgsql-hackers@lists.postgresql.org, peter.eisentraut@enterprisedb.com, pavel.stehule@gmail.com, samokhvalov@gmail.com
Subject: Re: [PATCH] Query Jumbling for CALL and SET utility statements
In-Reply-To: <09c04aba-7f12-e524-20f7-3f92c1479fdf@amazon.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox