public inbox for [email protected]  
help / color / mirror / Atom feed
[PATCH v52 5/7] Doc part of shared-memory based stats collector.
5+ messages / 4 participants
[nested] [flat]

* [PATCH v52 5/7] Doc part of shared-memory based stats collector.
@ 2020-03-19 06:11  Kyotaro Horiguchi <[email protected]>
  0 siblings, 0 replies; 5+ messages in thread

From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)

---
 doc/src/sgml/catalogs.sgml          |   6 +-
 doc/src/sgml/config.sgml            |  27 +++---
 doc/src/sgml/high-availability.sgml |  13 +--
 doc/src/sgml/monitoring.sgml        | 127 +++++++++++++---------------
 doc/src/sgml/ref/pg_dump.sgml       |   9 +-
 src/backend/postmaster/postmaster.c |   2 -
 6 files changed, 86 insertions(+), 98 deletions(-)

diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0674..0ef684d4d0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9261,9 +9261,9 @@ SCRAM-SHA-256$<replaceable>&lt;iteration count&gt;</replaceable>:<replaceable>&l
   <para>
    <xref linkend="view-table"/> lists the system views described here.
    More detailed documentation of each view follows below.
-   There are some additional views that provide access to the results of
-   the statistics collector; they are described in <xref
-   linkend="monitoring-stats-views-table"/>.
+   There are some additional views that provide access to the activity
+   statistics; they are described in
+   <xref linkend="monitoring-stats-views-table"/>.
   </para>
 
   <para>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 529876895b..99a8664026 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7375,11 +7375,11 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
     <title>Run-time Statistics</title>
 
     <sect2 id="runtime-config-statistics-collector">
-     <title>Query and Index Statistics Collector</title>
+     <title>Query and Index Activity Statistics</title>
 
      <para>
-      These parameters control server-wide statistics collection features.
-      When statistics collection is enabled, the data that is produced can be
+      These parameters control server-wide activity statistics features.
+      When activity statistics is enabled, the data that is produced can be
       accessed via the <structname>pg_stat</structname> and
       <structname>pg_statio</structname> family of system views.
       Refer to <xref linkend="monitoring"/> for more information.
@@ -7395,14 +7395,13 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       </term>
       <listitem>
        <para>
-        Enables the collection of information on the currently
-        executing command of each session, along with the time when
-        that command began execution. This parameter is on by
-        default. Note that even when enabled, this information is not
-        visible to all users, only to superusers and the user owning
-        the session being reported on, so it should not represent a
-        security risk.
-        Only superusers can change this setting.
+        Enables activity tracking on the currently executing command of
+        each session, along with the time when that command began
+        execution. This parameter is on by default. Note that even when
+        enabled, this information is not visible to all users, only to
+        superusers and the user owning the session being reported on, so it
+        should not represent a security risk.  Only superusers can change this
+        setting.
        </para>
       </listitem>
      </varlistentry>
@@ -7433,9 +7432,9 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       </term>
       <listitem>
        <para>
-        Enables collection of statistics on database activity.
+        Enables tracking of database activity.
         This parameter is on by default, because the autovacuum
-        daemon needs the collected information.
+        daemon needs the activity information.
         Only superusers can change this setting.
        </para>
       </listitem>
@@ -8554,7 +8553,7 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       <listitem>
        <para>
         Specifies the fraction of the total number of heap tuples counted in
-        the previous statistics collection that can be inserted without
+        the previously collected statistics that can be inserted without
         incurring an index scan at the <command>VACUUM</command> cleanup stage.
         This setting currently applies to B-tree indexes only.
        </para>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index f49f5c0108..45095857eb 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -2217,12 +2217,13 @@ HINT:  You can then restart the server after making the necessary configuration
    </para>
 
    <para>
-    The statistics collector is active during recovery. All scans, reads, blocks,
-    index usage, etc., will be recorded normally on the standby. Replayed
-    actions will not duplicate their effects on primary, so replaying an
-    insert will not increment the Inserts column of pg_stat_user_tables.
-    The stats file is deleted at the start of recovery, so stats from primary
-    and standby will differ; this is considered a feature, not a bug.
+    The activity statistics is collected during recovery. All scans, reads,
+    blocks, index usage, etc., will be recorded normally on the
+    standby. Replayed actions will not duplicate their effects on primary, so
+    replaying an insert will not increment the Inserts column of
+    pg_stat_user_tables.  The activity statistics is reset at the start of
+    recovery, so stats from primary and standby will differ; this is
+    considered a feature, not a bug.
    </para>
 
    <para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3335d71eba..6217ed9792 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -22,7 +22,7 @@
   <para>
    Several tools are available for monitoring database activity and
    analyzing performance.  Most of this chapter is devoted to describing
-   <productname>PostgreSQL</productname>'s statistics collector,
+   <productname>PostgreSQL</productname>'s activity statistics,
    but one should not neglect regular Unix monitoring programs such as
    <command>ps</command>, <command>top</command>, <command>iostat</command>, and <command>vmstat</command>.
    Also, once one has identified a
@@ -53,7 +53,6 @@ postgres  15554  0.0  0.0  57536  1184 ?        Ss   18:02   0:00 postgres: back
 postgres  15555  0.0  0.0  57536   916 ?        Ss   18:02   0:00 postgres: checkpointer
 postgres  15556  0.0  0.0  57536   916 ?        Ss   18:02   0:00 postgres: walwriter
 postgres  15557  0.0  0.0  58504  2244 ?        Ss   18:02   0:00 postgres: autovacuum launcher
-postgres  15558  0.0  0.0  17512  1068 ?        Ss   18:02   0:00 postgres: stats collector
 postgres  15582  0.0  0.0  58772  3080 ?        Ss   18:04   0:00 postgres: joe runbug 127.0.0.1 idle
 postgres  15606  0.0  0.0  58772  3052 ?        Ss   18:07   0:00 postgres: tgl regression [local] SELECT waiting
 postgres  15610  0.0  0.0  58772  3056 ?        Ss   18:07   0:00 postgres: tgl regression [local] idle in transaction
@@ -65,9 +64,8 @@ postgres  15610  0.0  0.0  58772  3056 ?        Ss   18:07   0:00 postgres: tgl
    primary server process.  The command arguments
    shown for it are the same ones used when it was launched.  The next five
    processes are background worker processes automatically launched by the
-   primary process.  (The <quote>stats collector</quote> process will not be present
-   if you have set the system not to start the statistics collector; likewise
-   the <quote>autovacuum launcher</quote> process can be disabled.)
+   primary process.  (The <quote>autovacuum launcher</quote> process will not
+   be present if you have set the system not to start it.)
    Each of the remaining
    processes is a server process handling one client connection.  Each such
    process sets its command line display in the form
@@ -130,20 +128,21 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
  </sect1>
 
  <sect1 id="monitoring-stats">
-  <title>The Statistics Collector</title>
+  <title>The Activity Statistics</title>
 
   <indexterm zone="monitoring-stats">
    <primary>statistics</primary>
   </indexterm>
 
   <para>
-   <productname>PostgreSQL</productname>'s <firstterm>statistics collector</firstterm>
-   is a subsystem that supports collection and reporting of information about
-   server activity.  Presently, the collector can count accesses to tables
-   and indexes in both disk-block and individual-row terms.  It also tracks
-   the total number of rows in each table, and information about vacuum and
-   analyze actions for each table.  It can also count calls to user-defined
-   functions and the total time spent in each one.
+   <productname>PostgreSQL</productname>'s <firstterm>activity
+   statistics</firstterm> is a subsystem that supports tracking and reporting
+   of information about server activity.  Presently, the activity statistics
+   tracks the count of accesses to tables and indexes in both disk-block and
+   individual-row terms.  It also tracks the total number of rows in each
+   table, and information about vacuum and analyze actions for each table.  It
+   can also track calls to user-defined functions and the total time spent in
+   each one.
   </para>
 
   <para>
@@ -151,15 +150,15 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
    information about exactly what is going on in the system right now, such as
    the exact command currently being executed by other server processes, and
    which other connections exist in the system.  This facility is independent
-   of the collector process.
+   of the activity statistics.
   </para>
 
  <sect2 id="monitoring-stats-setup">
-  <title>Statistics Collection Configuration</title>
+  <title>Activity Statistics Configuration</title>
 
   <para>
-   Since collection of statistics adds some overhead to query execution,
-   the system can be configured to collect or not collect information.
+   Since tracking for the activity statistics adds some overhead to query
+   execution, the system can be configured to track or not track activity.
    This is controlled by configuration parameters that are normally set in
    <filename>postgresql.conf</filename>.  (See <xref linkend="runtime-config"/> for
    details about setting configuration parameters.)
@@ -172,7 +171,7 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
 
   <para>
    The parameter <xref linkend="guc-track-counts"/> controls whether
-   statistics are collected about table and index accesses.
+   to track activity about table and index accesses.
   </para>
 
   <para>
@@ -201,18 +200,11 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
   </para>
 
   <para>
-   The statistics collector transmits the collected information to other
-   <productname>PostgreSQL</productname> processes through temporary files.
-   These files are stored in the directory named by the
-   <xref linkend="guc-stats-temp-directory"/> parameter,
-   <filename>pg_stat_tmp</filename> by default.
-   For better performance, <varname>stats_temp_directory</varname> can be
-   pointed at a RAM-based file system, decreasing physical I/O requirements.
-   When the server shuts down cleanly, a permanent copy of the statistics
-   data is stored in the <filename>pg_stat</filename> subdirectory, so that
-   statistics can be retained across server restarts.  When recovery is
-   performed at server start (e.g., after immediate shutdown, server crash,
-   and point-in-time recovery), all statistics counters are reset.
+   down cleanly, a permanent copy of the statistics data is stored in
+   the <filename>pg_stat</filename> subdirectory, so that statistics can be
+   retained across server restarts.  When recovery is performed at server
+   start (e.g. after immediate shutdown, server crash, and point-in-time
+   recovery), all statistics counters are reset.
   </para>
 
  </sect2>
@@ -225,48 +217,46 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
    linkend="monitoring-stats-dynamic-views-table"/>, are available to show
    the current state of the system. There are also several other
    views, listed in <xref
-   linkend="monitoring-stats-views-table"/>, available to show the results
-   of statistics collection.  Alternatively, one can
-   build custom views using the underlying statistics functions, as discussed
-   in <xref linkend="monitoring-stats-functions"/>.
+   linkend="monitoring-stats-views-table"/>, available to show the activity
+   statistics.  Alternatively, one can build custom views using the underlying
+   statistics functions, as discussed in
+   <xref linkend="monitoring-stats-functions"/>.
   </para>
 
   <para>
-   When using the statistics to monitor collected data, it is important
-   to realize that the information does not update instantaneously.
-   Each individual server process transmits new statistical counts to
-   the collector just before going idle; so a query or transaction still in
-   progress does not affect the displayed totals.  Also, the collector itself
-   emits a new report at most once per <varname>PGSTAT_STAT_INTERVAL</varname>
-   milliseconds (500 ms unless altered while building the server).  So the
-   displayed information lags behind actual activity.  However, current-query
-   information collected by <varname>track_activities</varname> is
-   always up-to-date.
+   When using the activity statistics, it is important to realize that the
+   information does not update instantaneously.  Each individual server writes
+   out new statistical counts just before going idle, not frequent than once
+   per <varname>PGSTAT_STAT_INTERVAL</varname> milliseconds (1 second unless
+   altered while building the server); so a query or transaction still in
+   progress does not affect the displayed totals.  However, current-query
+   information tracked by <varname>track_activities</varname> is always
+   up-to-date.
   </para>
 
   <para>
    Another important point is that when a server process is asked to display
-   any of these statistics, it first fetches the most recent report emitted by
-   the collector process and then continues to use this snapshot for all
-   statistical views and functions until the end of its current transaction.
-   So the statistics will show static information as long as you continue the
-   current transaction.  Similarly, information about the current queries of
-   all sessions is collected when any such information is first requested
-   within a transaction, and the same information will be displayed throughout
-   the transaction.
-   This is a feature, not a bug, because it allows you to perform several
-   queries on the statistics and correlate the results without worrying that
-   the numbers are changing underneath you.  But if you want to see new
-   results with each query, be sure to do the queries outside any transaction
-   block.  Alternatively, you can invoke
+   any of these statistics, it first reads the current statistics and then
+   continues to use this snapshot for all statistical views and functions
+   until the end of its current transaction.  So the statistics will show
+   static information as long as you continue the current transaction.
+   Similarly, information about the current queries of all sessions is tracked
+   when any such information is first requested within a transaction, and the
+   same information will be displayed throughout the transaction.  This is a
+   feature, not a bug, because it allows you to perform several queries on the
+   statistics and correlate the results without worrying that the numbers are
+   changing underneath you.  But if you want to see new results with each
+   query, be sure to do the queries outside any transaction block.
+   Alternatively, you can invoke
    <function>pg_stat_clear_snapshot</function>(), which will discard the
    current transaction's statistics snapshot (if any).  The next use of
    statistical information will cause a new snapshot to be fetched.
   </para>
-
+  
   <para>
-   A transaction can also see its own statistics (as yet untransmitted to the
-   collector) in the views <structname>pg_stat_xact_all_tables</structname>,
+   A transaction can also see its own statistics (as yet unwritten to the
+   server-wide activity statistics) in the
+   views <structname>pg_stat_xact_all_tables</structname>,
    <structname>pg_stat_xact_sys_tables</structname>,
    <structname>pg_stat_xact_user_tables</structname>, and
    <structname>pg_stat_xact_user_functions</structname>.  These numbers do not act as
@@ -648,7 +638,7 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
    kernel's I/O cache, and might therefore still be fetched without
    requiring a physical read. Users interested in obtaining more
    detailed information on <productname>PostgreSQL</productname> I/O behavior are
-   advised to use the <productname>PostgreSQL</productname> statistics collector
+   advised to use the <productname>PostgreSQL</productname> activity statistics
    in combination with operating system utilities that allow insight
    into the kernel's handling of I/O.
   </para>
@@ -1085,10 +1075,6 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
       <entry><literal>LogicalLauncherMain</literal></entry>
       <entry>Waiting in main loop of logical replication launcher process.</entry>
      </row>
-     <row>
-      <entry><literal>PgStatMain</literal></entry>
-      <entry>Waiting in main loop of statistics collector process.</entry>
-     </row>
      <row>
       <entry><literal>RecoveryWalStream</literal></entry>
       <entry>Waiting in main loop of startup process for WAL to arrive, during
@@ -1843,6 +1829,10 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
     </thead>
 
     <tbody>
+     <row>
+      <entry><literal>ActivityStatistics</literal></entry>
+      <entry>Waiting to write out activity statistics to shared memory.</entry>
+     </row>
      <row>
       <entry><literal>AddinShmemInit</literal></entry>
       <entry>Waiting to manage an extension's space allocation in shared
@@ -6123,9 +6113,10 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid,
      <entry><literal>performing final cleanup</literal></entry>
      <entry>
        <command>VACUUM</command> is performing final cleanup.  During this phase,
-       <command>VACUUM</command> will vacuum the free space map, update statistics
-       in <literal>pg_class</literal>, and report statistics to the statistics
-       collector.  When this phase is completed, <command>VACUUM</command> will end.
+       <command>VACUUM</command> will vacuum the free space map, update
+       statistics in <literal>pg_class</literal>, and system-wide activity
+       statistics.  When this phase is completed, <command>VACUUM</command>
+       will end.
      </entry>
     </row>
    </tbody>
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..1fa59a2fdf 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1280,11 +1280,10 @@ PostgreSQL documentation
   </para>
 
   <para>
-   The database activity of <application>pg_dump</application> is
-   normally collected by the statistics collector.  If this is
-   undesirable, you can set parameter <varname>track_counts</varname>
-   to false via <envar>PGOPTIONS</envar> or the <literal>ALTER
-   USER</literal> command.
+   The database activity of <application>pg_dump</application> is normally
+   collected.  If this is undesirable, you can set
+   parameter <varname>track_counts</varname> to false
+   via <envar>PGOPTIONS</envar> or the <literal>ALTER USER</literal> command.
   </para>
 
  </refsect1>
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 0abc41c33c..d74b5eed82 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -6328,8 +6328,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
 	if (postmaster_alive_fds[1] >= 0)
 		ReserveExternalFD();
 #endif
-	if (pgStatSock != PGINVALID_SOCKET)
-		ReserveExternalFD();
 }
 
 
-- 
2.27.0


----Next_Part(Wed_Mar_10_17_51_37_2021_192)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="v52-0006-Remove-the-GUC-stats_temp_directory.patch"



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry
@ 2023-02-09 10:38  Drouvot, Bertrand <[email protected]>
  0 siblings, 2 replies; 5+ messages in thread

From: Drouvot, Bertrand @ 2023-02-09 10:38 UTC (permalink / raw)
  To: PostgreSQL Hackers <[email protected]>

Hi hackers,

Please find attached a patch proposal for $SUBJECT.

The idea has been raised in [1] by Andres: it would allow to simplify even more the work done to
generate pg_stat_get_xact*() functions with Macros.

Indeed, with the reconciliation done in find_tabstat_entry() then all the pg_stat_get_xact*() functions
(making use of find_tabstat_entry()) now "look the same" (should they take into account live subtransactions or not).

Looking forward to your feedback,

Regards

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

[1]: https://www.postgresql.org/message-id/20230111225907.6el6c5j3hukizqxc%40awork3.anarazel.de
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 0fa5370bcd..a08a2c8e66 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -291,7 +291,7 @@ static const PgStat_KindInfo pgstat_kind_infos[PGSTAT_NUM_KINDS] = {
 		.shared_size = sizeof(PgStatShared_Function),
 		.shared_data_off = offsetof(PgStatShared_Function, stats),
 		.shared_data_len = sizeof(((PgStatShared_Function *) 0)->stats),
-		.pending_size = sizeof(PgStat_BackendFunctionEntry),
+		.pending_size = sizeof(PgStat_FunctionCounts),
 
 		.flush_pending_cb = pgstat_function_flush_cb,
 	},
diff --git a/src/backend/utils/activity/pgstat_function.c b/src/backend/utils/activity/pgstat_function.c
index 6139a27fee..0fdcefb783 100644
--- a/src/backend/utils/activity/pgstat_function.c
+++ b/src/backend/utils/activity/pgstat_function.c
@@ -73,7 +73,7 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
 						   PgStat_FunctionCallUsage *fcu)
 {
 	PgStat_EntryRef *entry_ref;
-	PgStat_BackendFunctionEntry *pending;
+	PgStat_FunctionCounts *pending;
 	bool		created_entry;
 
 	if (pgstat_track_functions <= fcinfo->flinfo->fn_stats)
@@ -121,10 +121,10 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
 
 	pending = entry_ref->pending;
 
-	fcu->fs = &pending->f_counts;
+	fcu->fs = pending;
 
 	/* save stats for this function, later used to compensate for recursion */
-	fcu->save_f_total_time = pending->f_counts.f_total_time;
+	fcu->save_f_total_time = pending->f_total_time;
 
 	/* save current backend-wide total time */
 	fcu->save_total = total_func_time;
@@ -192,10 +192,10 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
 bool
 pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 {
-	PgStat_BackendFunctionEntry *localent;
+	PgStat_FunctionCounts *localent;
 	PgStatShared_Function *shfuncent;
 
-	localent = (PgStat_BackendFunctionEntry *) entry_ref->pending;
+	localent = (PgStat_FunctionCounts *) entry_ref->pending;
 	shfuncent = (PgStatShared_Function *) entry_ref->shared_stats;
 
 	/* localent always has non-zero content */
@@ -203,11 +203,11 @@ pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 	if (!pgstat_lock_entry(entry_ref, nowait))
 		return false;
 
-	shfuncent->stats.f_numcalls += localent->f_counts.f_numcalls;
+	shfuncent->stats.f_numcalls += localent->f_numcalls;
 	shfuncent->stats.f_total_time +=
-		INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+		INSTR_TIME_GET_MICROSEC(localent->f_total_time);
 	shfuncent->stats.f_self_time +=
-		INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+		INSTR_TIME_GET_MICROSEC(localent->f_self_time);
 
 	pgstat_unlock_entry(entry_ref);
 
@@ -215,11 +215,11 @@ pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 }
 
 /*
- * find any existing PgStat_BackendFunctionEntry entry for specified function
+ * find any existing PgStat_FunctionCounts entry for specified function
  *
  * If no entry, return NULL, don't create a new one
  */
-PgStat_BackendFunctionEntry *
+PgStat_FunctionCounts *
 find_funcstat_entry(Oid func_id)
 {
 	PgStat_EntryRef *entry_ref;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2e20b93c20..b33965c581 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -466,14 +466,30 @@ PgStat_TableStatus *
 find_tabstat_entry(Oid rel_id)
 {
 	PgStat_EntryRef *entry_ref;
+	PgStat_TableStatus *tablestatus = NULL;
 
 	entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, MyDatabaseId, rel_id);
 	if (!entry_ref)
 		entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, InvalidOid, rel_id);
 
 	if (entry_ref)
-		return entry_ref->pending;
-	return NULL;
+	{
+		PgStat_TableStatus *tabentry = (PgStat_TableStatus *) entry_ref->pending;
+		PgStat_TableXactStatus *trans;
+
+		tablestatus  = palloc(sizeof(PgStat_TableStatus));
+		memcpy(tablestatus, tabentry, sizeof(PgStat_TableStatus));
+
+		/* live subtransactions' counts aren't in t_counts yet */
+		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
+		{
+			tablestatus->t_counts.t_tuples_inserted += trans->tuples_inserted;
+			tablestatus->t_counts.t_tuples_updated += trans->tuples_updated;
+			tablestatus->t_counts.t_tuples_deleted += trans->tuples_deleted;
+		}
+	}
+
+	return tablestatus;
 }
 
 /*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6737493402..40a6fbf871 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1366,7 +1366,10 @@ pg_stat_get_xact_numscans(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_numscans);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1381,7 +1384,10 @@ pg_stat_get_xact_tuples_returned(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_returned);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1396,7 +1402,10 @@ pg_stat_get_xact_tuples_fetched(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_fetched);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1407,16 +1416,13 @@ pg_stat_get_xact_tuples_inserted(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_inserted;
-		/* live subtransactions' counts aren't in t_tuples_inserted yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_inserted;
+		result = (int64) (tabentry->t_counts.t_tuples_inserted);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1428,16 +1434,13 @@ pg_stat_get_xact_tuples_updated(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_updated;
-		/* live subtransactions' counts aren't in t_tuples_updated yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_updated;
+		result = (int64) (tabentry->t_counts.t_tuples_updated);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1449,16 +1452,13 @@ pg_stat_get_xact_tuples_deleted(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_deleted;
-		/* live subtransactions' counts aren't in t_tuples_deleted yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_deleted;
+		result = (int64) (tabentry->t_counts.t_tuples_deleted);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1474,7 +1474,10 @@ pg_stat_get_xact_tuples_hot_updated(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_hot_updated);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1489,7 +1492,10 @@ pg_stat_get_xact_blocks_fetched(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_blocks_fetched);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1504,7 +1510,10 @@ pg_stat_get_xact_blocks_hit(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_blocks_hit);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1513,33 +1522,33 @@ Datum
 pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_INT64(funcentry->f_counts.f_numcalls);
+	PG_RETURN_INT64(funcentry->f_numcalls);
 }
 
 Datum
 pg_stat_get_xact_function_total_time(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_counts.f_total_time));
+	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_total_time));
 }
 
 Datum
 pg_stat_get_xact_function_self_time(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_counts.f_self_time));
+	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_self_time));
 }
 
 
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 5e3326a3b9..b94ae7f70d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -110,15 +110,6 @@ typedef struct PgStat_FunctionCounts
 	instr_time	f_self_time;
 } PgStat_FunctionCounts;
 
-/* ----------
- * PgStat_BackendFunctionEntry	Non-flushed function stats.
- * ----------
- */
-typedef struct PgStat_BackendFunctionEntry
-{
-	PgStat_FunctionCounts f_counts;
-} PgStat_BackendFunctionEntry;
-
 /*
  * Working state needed to accumulate per-function-call timing statistics.
  */
@@ -491,7 +482,7 @@ extern void pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu,
 									  bool finalize);
 
 extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid func_id);
-extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
+extern PgStat_FunctionCounts *find_funcstat_entry(Oid func_id);
 
 
 /*


Attachments:

  [text/plain] v1-0001-find_tabstat_entry_recon_get_rid_PgStat_BackendFunctionEntry.patch (9.9K, ../../[email protected]/2-v1-0001-find_tabstat_entry_recon_get_rid_PgStat_BackendFunctionEntry.patch)
  download | inline diff:
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index 0fa5370bcd..a08a2c8e66 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -291,7 +291,7 @@ static const PgStat_KindInfo pgstat_kind_infos[PGSTAT_NUM_KINDS] = {
 		.shared_size = sizeof(PgStatShared_Function),
 		.shared_data_off = offsetof(PgStatShared_Function, stats),
 		.shared_data_len = sizeof(((PgStatShared_Function *) 0)->stats),
-		.pending_size = sizeof(PgStat_BackendFunctionEntry),
+		.pending_size = sizeof(PgStat_FunctionCounts),
 
 		.flush_pending_cb = pgstat_function_flush_cb,
 	},
diff --git a/src/backend/utils/activity/pgstat_function.c b/src/backend/utils/activity/pgstat_function.c
index 6139a27fee..0fdcefb783 100644
--- a/src/backend/utils/activity/pgstat_function.c
+++ b/src/backend/utils/activity/pgstat_function.c
@@ -73,7 +73,7 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
 						   PgStat_FunctionCallUsage *fcu)
 {
 	PgStat_EntryRef *entry_ref;
-	PgStat_BackendFunctionEntry *pending;
+	PgStat_FunctionCounts *pending;
 	bool		created_entry;
 
 	if (pgstat_track_functions <= fcinfo->flinfo->fn_stats)
@@ -121,10 +121,10 @@ pgstat_init_function_usage(FunctionCallInfo fcinfo,
 
 	pending = entry_ref->pending;
 
-	fcu->fs = &pending->f_counts;
+	fcu->fs = pending;
 
 	/* save stats for this function, later used to compensate for recursion */
-	fcu->save_f_total_time = pending->f_counts.f_total_time;
+	fcu->save_f_total_time = pending->f_total_time;
 
 	/* save current backend-wide total time */
 	fcu->save_total = total_func_time;
@@ -192,10 +192,10 @@ pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu, bool finalize)
 bool
 pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 {
-	PgStat_BackendFunctionEntry *localent;
+	PgStat_FunctionCounts *localent;
 	PgStatShared_Function *shfuncent;
 
-	localent = (PgStat_BackendFunctionEntry *) entry_ref->pending;
+	localent = (PgStat_FunctionCounts *) entry_ref->pending;
 	shfuncent = (PgStatShared_Function *) entry_ref->shared_stats;
 
 	/* localent always has non-zero content */
@@ -203,11 +203,11 @@ pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 	if (!pgstat_lock_entry(entry_ref, nowait))
 		return false;
 
-	shfuncent->stats.f_numcalls += localent->f_counts.f_numcalls;
+	shfuncent->stats.f_numcalls += localent->f_numcalls;
 	shfuncent->stats.f_total_time +=
-		INSTR_TIME_GET_MICROSEC(localent->f_counts.f_total_time);
+		INSTR_TIME_GET_MICROSEC(localent->f_total_time);
 	shfuncent->stats.f_self_time +=
-		INSTR_TIME_GET_MICROSEC(localent->f_counts.f_self_time);
+		INSTR_TIME_GET_MICROSEC(localent->f_self_time);
 
 	pgstat_unlock_entry(entry_ref);
 
@@ -215,11 +215,11 @@ pgstat_function_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
 }
 
 /*
- * find any existing PgStat_BackendFunctionEntry entry for specified function
+ * find any existing PgStat_FunctionCounts entry for specified function
  *
  * If no entry, return NULL, don't create a new one
  */
-PgStat_BackendFunctionEntry *
+PgStat_FunctionCounts *
 find_funcstat_entry(Oid func_id)
 {
 	PgStat_EntryRef *entry_ref;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 2e20b93c20..b33965c581 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -466,14 +466,30 @@ PgStat_TableStatus *
 find_tabstat_entry(Oid rel_id)
 {
 	PgStat_EntryRef *entry_ref;
+	PgStat_TableStatus *tablestatus = NULL;
 
 	entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, MyDatabaseId, rel_id);
 	if (!entry_ref)
 		entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, InvalidOid, rel_id);
 
 	if (entry_ref)
-		return entry_ref->pending;
-	return NULL;
+	{
+		PgStat_TableStatus *tabentry = (PgStat_TableStatus *) entry_ref->pending;
+		PgStat_TableXactStatus *trans;
+
+		tablestatus  = palloc(sizeof(PgStat_TableStatus));
+		memcpy(tablestatus, tabentry, sizeof(PgStat_TableStatus));
+
+		/* live subtransactions' counts aren't in t_counts yet */
+		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
+		{
+			tablestatus->t_counts.t_tuples_inserted += trans->tuples_inserted;
+			tablestatus->t_counts.t_tuples_updated += trans->tuples_updated;
+			tablestatus->t_counts.t_tuples_deleted += trans->tuples_deleted;
+		}
+	}
+
+	return tablestatus;
 }
 
 /*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6737493402..40a6fbf871 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1366,7 +1366,10 @@ pg_stat_get_xact_numscans(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_numscans);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1381,7 +1384,10 @@ pg_stat_get_xact_tuples_returned(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_returned);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1396,7 +1402,10 @@ pg_stat_get_xact_tuples_fetched(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_fetched);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1407,16 +1416,13 @@ pg_stat_get_xact_tuples_inserted(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_inserted;
-		/* live subtransactions' counts aren't in t_tuples_inserted yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_inserted;
+		result = (int64) (tabentry->t_counts.t_tuples_inserted);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1428,16 +1434,13 @@ pg_stat_get_xact_tuples_updated(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_updated;
-		/* live subtransactions' counts aren't in t_tuples_updated yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_updated;
+		result = (int64) (tabentry->t_counts.t_tuples_updated);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1449,16 +1452,13 @@ pg_stat_get_xact_tuples_deleted(PG_FUNCTION_ARGS)
 	Oid			relid = PG_GETARG_OID(0);
 	int64		result;
 	PgStat_TableStatus *tabentry;
-	PgStat_TableXactStatus *trans;
 
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
 	{
-		result = tabentry->t_counts.t_tuples_deleted;
-		/* live subtransactions' counts aren't in t_tuples_deleted yet */
-		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
-			result += trans->tuples_deleted;
+		result = (int64) (tabentry->t_counts.t_tuples_deleted);
+		pfree(tabentry);
 	}
 
 	PG_RETURN_INT64(result);
@@ -1474,7 +1474,10 @@ pg_stat_get_xact_tuples_hot_updated(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_tuples_hot_updated);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1489,7 +1492,10 @@ pg_stat_get_xact_blocks_fetched(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_blocks_fetched);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1504,7 +1510,10 @@ pg_stat_get_xact_blocks_hit(PG_FUNCTION_ARGS)
 	if ((tabentry = find_tabstat_entry(relid)) == NULL)
 		result = 0;
 	else
+	{
 		result = (int64) (tabentry->t_counts.t_blocks_hit);
+		pfree(tabentry);
+	}
 
 	PG_RETURN_INT64(result);
 }
@@ -1513,33 +1522,33 @@ Datum
 pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_INT64(funcentry->f_counts.f_numcalls);
+	PG_RETURN_INT64(funcentry->f_numcalls);
 }
 
 Datum
 pg_stat_get_xact_function_total_time(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_counts.f_total_time));
+	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_total_time));
 }
 
 Datum
 pg_stat_get_xact_function_self_time(PG_FUNCTION_ARGS)
 {
 	Oid			funcid = PG_GETARG_OID(0);
-	PgStat_BackendFunctionEntry *funcentry;
+	PgStat_FunctionCounts *funcentry;
 
 	if ((funcentry = find_funcstat_entry(funcid)) == NULL)
 		PG_RETURN_NULL();
-	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_counts.f_self_time));
+	PG_RETURN_FLOAT8(INSTR_TIME_GET_MILLISEC(funcentry->f_self_time));
 }
 
 
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 5e3326a3b9..b94ae7f70d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -110,15 +110,6 @@ typedef struct PgStat_FunctionCounts
 	instr_time	f_self_time;
 } PgStat_FunctionCounts;
 
-/* ----------
- * PgStat_BackendFunctionEntry	Non-flushed function stats.
- * ----------
- */
-typedef struct PgStat_BackendFunctionEntry
-{
-	PgStat_FunctionCounts f_counts;
-} PgStat_BackendFunctionEntry;
-
 /*
  * Working state needed to accumulate per-function-call timing statistics.
  */
@@ -491,7 +482,7 @@ extern void pgstat_end_function_usage(PgStat_FunctionCallUsage *fcu,
 									  bool finalize);
 
 extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid func_id);
-extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
+extern PgStat_FunctionCounts *find_funcstat_entry(Oid func_id);
 
 
 /*


^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry
@ 2023-02-10 02:32  Kyotaro Horiguchi <[email protected]>
  parent: Drouvot, Bertrand <[email protected]>
  1 sibling, 1 reply; 5+ messages in thread

From: Kyotaro Horiguchi @ 2023-02-10 02:32 UTC (permalink / raw)
  To: [email protected]; +Cc: [email protected]

At Thu, 9 Feb 2023 11:38:18 +0100, "Drouvot, Bertrand" <[email protected]> wrote in 
> Hi hackers,
> 
> Please find attached a patch proposal for $SUBJECT.
> 
> The idea has been raised in [1] by Andres: it would allow to simplify
> even more the work done to
> generate pg_stat_get_xact*() functions with Macros.
> 
> Indeed, with the reconciliation done in find_tabstat_entry() then all
> the pg_stat_get_xact*() functions
> (making use of find_tabstat_entry()) now "look the same" (should they
> take into account live subtransactions or not).
> 
> Looking forward to your feedback,

I like that direction.

Don't we rename PgStat_FunctionCounts to PgStat_FuncStatus, unifying
neighboring functions?

Why does find_tabstat_entry() copies the whole pending data and
performs subxaction summarization? The summarization is needed only by
few callers but now that cost is imposed to the all callers along with
additional palloc()/pfree() calls. That doesn't seem reasonable.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center






^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry
@ 2023-02-10 15:50  Drouvot, Bertrand <[email protected]>
  parent: Kyotaro Horiguchi <[email protected]>
  0 siblings, 0 replies; 5+ messages in thread

From: Drouvot, Bertrand @ 2023-02-10 15:50 UTC (permalink / raw)
  To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]

Hi,

On 2/10/23 3:32 AM, Kyotaro Horiguchi wrote:
> At Thu, 9 Feb 2023 11:38:18 +0100, "Drouvot, Bertrand" <[email protected]> wrote in
>> Hi hackers,
>>
>> Please find attached a patch proposal for $SUBJECT.
>>
>> The idea has been raised in [1] by Andres: it would allow to simplify
>> even more the work done to
>> generate pg_stat_get_xact*() functions with Macros.
>>
>> Indeed, with the reconciliation done in find_tabstat_entry() then all
>> the pg_stat_get_xact*() functions
>> (making use of find_tabstat_entry()) now "look the same" (should they
>> take into account live subtransactions or not).
>>
>> Looking forward to your feedback,
> 
> I like that direction.
> 

Thanks for looking at it!

> Don't we rename PgStat_FunctionCounts to PgStat_FuncStatus, unifying
> neighboring functions?
> 

Not sure, I think it's the counter part of PgStat_TableCounts for example.

> Why does find_tabstat_entry() copies the whole pending data and
> performs subxaction summarization? 

It copies the pending data to not increment it's counters while doing the summarization.
The summarization was done here to avoid the pg_stat_get_xact*() functions to do the computation so that all
the pg_stat_get_xact*() functions look the same but....

> The summarization is needed only by
> few callers but now that cost is imposed to the all callers along with
> additional palloc()/pfree() calls. That doesn't seem reasonable.
> 

I agree that's not the best approach.....

Let me come back with another proposal (thinking to increment reconciled
counters in pgstat_count_heap_insert(), pgstat_count_heap_delete() and
pgstat_count_heap_update()).

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com






^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry
@ 2023-02-10 21:46  Andres Freund <[email protected]>
  parent: Drouvot, Bertrand <[email protected]>
  1 sibling, 0 replies; 5+ messages in thread

From: Andres Freund @ 2023-02-10 21:46 UTC (permalink / raw)
  To: Drouvot, Bertrand <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>

Hi,

On 2023-02-09 11:38:18 +0100, Drouvot, Bertrand wrote:
> Please find attached a patch proposal for $SUBJECT.
> 
> The idea has been raised in [1] by Andres: it would allow to simplify even more the work done to
> generate pg_stat_get_xact*() functions with Macros.

Thanks!

I think this is useful beyond being able to generate those functions with
macros. The fact that we had to deal with transactional code in pgstatfuncs.c
meant that a lot of the relevant itnernals had to be exposed "outside" pgstat,
which seems architecturally not great.


> Indeed, with the reconciliation done in find_tabstat_entry() then all the pg_stat_get_xact*() functions
> (making use of find_tabstat_entry()) now "look the same" (should they take into account live subtransactions or not).

I'm not bothered by making all of pg_stat_get_xact* functions more expensive,
they're not a hot code path. But if we need to, we could just add a parameter
to find_tabstat_entry() indicating whether we need to reconcile or not.


>  	/* save stats for this function, later used to compensate for recursion */
> -	fcu->save_f_total_time = pending->f_counts.f_total_time;
> +	fcu->save_f_total_time = pending->f_total_time;
>  
>  	/* save current backend-wide total time */
>  	fcu->save_total = total_func_time;

The diff is noisy due to all the mechanical changes like the above. Could that
be split into a separate commit?


>  find_tabstat_entry(Oid rel_id)
>  {
>  	PgStat_EntryRef *entry_ref;
> +	PgStat_TableStatus *tablestatus = NULL;
>  
>  	entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, MyDatabaseId, rel_id);
>  	if (!entry_ref)
>  		entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, InvalidOid, rel_id);
>  
>  	if (entry_ref)
> -		return entry_ref->pending;
> -	return NULL;
> +	{
> +		PgStat_TableStatus *tabentry = (PgStat_TableStatus *) entry_ref->pending;

I'd add an early return for the !entry_ref case, that way you don't need to
indent the bulk of the function.


> +		PgStat_TableXactStatus *trans;
> +
> +		tablestatus  = palloc(sizeof(PgStat_TableStatus));
> +		memcpy(tablestatus, tabentry, sizeof(PgStat_TableStatus));

For things like this I'd use
  *tablestatus = *tabentry;

that way the compiler will warn you about mismatching types, and you don't
need the sizeof().


> +		/* live subtransactions' counts aren't in t_counts yet */
> +		for (trans = tabentry->trans; trans != NULL; trans = trans->upper)
> +		{
> +			tablestatus->t_counts.t_tuples_inserted += trans->tuples_inserted;
> +			tablestatus->t_counts.t_tuples_updated += trans->tuples_updated;
> +			tablestatus->t_counts.t_tuples_deleted += trans->tuples_deleted;
> +		}
> +	}

Hm, why do we end uup with t_counts still being used here, but removed other
places?


> diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
> index 6737493402..40a6fbf871 100644
> --- a/src/backend/utils/adt/pgstatfuncs.c
> +++ b/src/backend/utils/adt/pgstatfuncs.c
> @@ -1366,7 +1366,10 @@ pg_stat_get_xact_numscans(PG_FUNCTION_ARGS)
>  	if ((tabentry = find_tabstat_entry(relid)) == NULL)
>  		result = 0;
>  	else
> +	{
>  		result = (int64) (tabentry->t_counts.t_numscans);
> +		pfree(tabentry);
> +	}
>  
>  	PG_RETURN_INT64(result);
>  }

I don't think we need to bother with individual pfrees in this path. The
caller will call the function in a dedicated memory context, that'll be reset
very soon after this.

Greetings,

Andres Freund






^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2023-02-10 21:46 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-19 06:11 [PATCH v52 5/7] Doc part of shared-memory based stats collector. Kyotaro Horiguchi <[email protected]>
2023-02-09 10:38 Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry Drouvot, Bertrand <[email protected]>
2023-02-10 02:32 ` Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry Kyotaro Horiguchi <[email protected]>
2023-02-10 15:50   ` Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry Drouvot, Bertrand <[email protected]>
2023-02-10 21:46 ` Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry Andres Freund <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox