From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@lists.postgresql.org, Sami Imseih <samimseih@gmail.com>
Subject: Re: Redesign per-backend statistics
Date: Mon, 26 Oct 2026 11:54:35 +0000
Message-ID: <at8/ezwmjHCWRmh2@bdtpg> (raw)
In-Reply-To: <arHy5b6v0s63faN4@bdtpg>
References: <anCjoFkGah/uB7lw@bdtpg>
<anU3OE2403PZdwJL@paquier.xyz>
<anXQM2PoJ+6iYYT2@bdtpg>
<anlVx_TD_Q8He5uf@paquier.xyz>
<annj/ETYOFgbrGsn@bdtpg>
<an3lxtz4TaYXXg/8@bdtpg>
<arDZoI/5xHU2utRh@bdtpg>
<fv5iqux4i5i6yveek2kz6v4rix3o7oyi2dlm7jkcarqb7pf4kk@fygc6ekt46ge>
<arHI3tQDbpnd2El2@paquier.xyz>
<arHy5b6v0s63faN4@bdtpg>
Hi,
On Tue, Sep 22, 2026 at 03:15:49AM +0000, Bertrand Drouvot wrote:
> Hi,
>
> On Tue, Sep 22, 2026 at 09:16:30AM +0900, Michael Paquier wrote:
> > On Mon, Sep 21, 2026 at 05:17:04AM -0400, Andres Freund wrote:
> > > On 2026-09-21 07:15:44 +0000, Bertrand Drouvot wrote:
> > >> +/* -------------------------------------------------------------------------
> > >> + *
> > >> + * pgstat_per_backend.c
> > >> + * Generic infrastructure for per-backend statistics.
> > >> + *
> > >> + * This file manages the dedicated per-kind dshashes used for per-backend
> > >> + * statistics, including entry creation, fetching, snapshots, transfer to
> > >> + * global statistics, and removal.
> > >
> > > It's not entirely obvious to me that a hash table is needed here. If I
> > > understand correctly, this just going to be used for per-backend versions of
> > > fixed stats. Which means we could just as well allocate all the memory in
> > > statically allocated shared memory at server start and have the per-backend
> > > stats be accessible by nothing more than an array access?
> >
> > Yeah. Array access based on an procnum index is still feeling like
> > the natural thing to do here (I think I've mentioned that upthread?).
>
> Yeah you did mention it. I took your reply in [1] as agreeing with the concern
> about reserving memory for mostly unused slots, though not necessarily with using
> a dshash. Is your preference still for the array despite that memory cost?
>
> [1]: https://postgr.es/m/anlVx_TD_Q8He5uf%40paquier.xyz
I had a chat with Andres about this thread at PGConf.EU last week, and I agree
that a fixed array is better than dshash. One reason is that dshash can also
consume a non negligible amount of memory.
For example, with 5000 maximum backends, the fixed array allocates about 16 MB.
Out of curiosity, I added some measurements to the dshash patch version and observed
that, with 3735 active client sessions, its memory usage jumps from 14 MB to 22 MB.
It was already about 10 MB with 1500 sessions.
So, PFA a new patch version making use of a fixed array and also adressing
Andres's comments upthread.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.comAttachments:download all (6 files, .tar.gz)v4-0001-pgstat-add-tests-for-per-backend-statistics.patch (text/x-diff, 18.7K) download
From 1d31d5f222c5dbaa563c9e3afee0adaa95163a7e Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Fri, 17 Jul 2026 13:23:40 +0000
Subject: [PATCH v4 1/6] pgstat: add tests for per-backend statistics
Add an isolation test for per-backend WAL statistics. Verify that SNAPSHOT,
CACHE and NONE modes behave as expected. Also verify that switching from CACHE
to SNAPSHOT discards previously cached entries and builds a fresh snapshot.
Use WAL as a representative statistics kind for consistency behavior rather
than repeating the same tests for every per-backend statistics kind.
Add a TAP test that starts a backend after another session has built a
statistics snapshot. Per-backend WAL, IO, and lock queries for the new backend
must return no rows rather than fabricated rows containing zeroes.
Finally, extend the core statistics regression test for per-backend WAL
reset. Check that resetting one backend reduces its counters and sets its
reset timestamp without removing that backend's activity from the global WAL
statistics.
These tests provide compatibility coverage for the per-backend redesign
changes introduced by the following commits.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
.../isolation/expected/stats-per-backend.out | 225 ++++++++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../isolation/specs/stats-per-backend.spec | 151 ++++++++++++
src/test/recovery/t/029_stats_restart.pl | 36 ++-
src/test/regress/expected/stats.out | 29 +++
src/test/regress/sql/stats.sql | 10 +
6 files changed, 450 insertions(+), 2 deletions(-)
46.0% src/test/isolation/expected/
35.3% src/test/isolation/specs/
9.2% src/test/recovery/t/
5.1% src/test/regress/expected/
3.9% src/test/regress/sql/
diff --git a/src/test/isolation/expected/stats-per-backend.out b/src/test/isolation/expected/stats-per-backend.outnew file mode 100644index 00000000000..5c8f3c990a1--- /dev/null+++ b/src/test/isolation/expected/stats-per-backend.out@@ -0,0 +1,225 @@+Parsed test spec with 2 sessions++starting permutation: s1_fetch_consistency_none s2_generate s1_begin s1_save_backend_stats s2_generate_more s1_check_uncached_backend s1_commit+step s1_fetch_consistency_none: SET stats_fetch_consistency = 'none';+step s2_generate: + INSERT INTO stats_per_backend_data VALUES (1);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_begin: BEGIN;+step s1_save_backend_stats: + INSERT INTO stats_per_backend_saved+ SELECT wal_records+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++step s2_generate_more: + INSERT INTO stats_per_backend_data VALUES (2);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_check_uncached_backend: + SELECT current.wal_records > saved.wal_records AS none_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;++none_is_refreshed+-----------------+t +(1 row)++step s1_commit: COMMIT;++starting permutation: s1_fetch_consistency_snapshot s1_begin s1_build_snapshot s2_generate s1_save_backend_stats s1_commit s1_check_live_backend+step s1_fetch_consistency_snapshot: SET stats_fetch_consistency = 'snapshot';+step s1_begin: BEGIN;+step s1_build_snapshot: + SELECT wal_records >= 0 AS snapshot_created FROM pg_stat_wal;++snapshot_created+----------------+t +(1 row)++step s2_generate: + INSERT INTO stats_per_backend_data VALUES (1);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_save_backend_stats: + INSERT INTO stats_per_backend_saved+ SELECT wal_records+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++step s1_commit: COMMIT;+step s1_check_live_backend: + SELECT current.wal_records > saved.wal_records AS snapshot_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;++snapshot_is_refreshed+---------------------+t +(1 row)+++starting permutation: s1_fetch_consistency_cache s2_generate s1_begin s1_save_backend_stats s2_generate_more s1_check_cached_backend s1_commit s1_check_refreshed_backend+step s1_fetch_consistency_cache: SET stats_fetch_consistency = 'cache';+step s2_generate: + INSERT INTO stats_per_backend_data VALUES (1);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_begin: BEGIN;+step s1_save_backend_stats: + INSERT INTO stats_per_backend_saved+ SELECT wal_records+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++step s2_generate_more: + INSERT INTO stats_per_backend_data VALUES (2);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_check_cached_backend: + SELECT current.wal_records = saved.wal_records AS cache_is_stable+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;++cache_is_stable+---------------+t +(1 row)++step s1_commit: COMMIT;+step s1_check_refreshed_backend: + SELECT current.wal_records > saved.wal_records AS cache_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;++cache_is_refreshed+------------------+t +(1 row)+++starting permutation: s1_fetch_consistency_cache s2_generate s1_begin s1_save_backend_stats s2_generate_more s1_fetch_consistency_snapshot s1_check_live_backend s1_commit+step s1_fetch_consistency_cache: SET stats_fetch_consistency = 'cache';+step s2_generate: + INSERT INTO stats_per_backend_data VALUES (1);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_begin: BEGIN;+step s1_save_backend_stats: + INSERT INTO stats_per_backend_saved+ SELECT wal_records+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++step s2_generate_more: + INSERT INTO stats_per_backend_data VALUES (2);+ SELECT pg_stat_force_next_flush();++pg_stat_force_next_flush+------------------------+ +(1 row)++step s1_fetch_consistency_snapshot: SET stats_fetch_consistency = 'snapshot';+step s1_check_live_backend: + SELECT current.wal_records > saved.wal_records AS snapshot_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;++snapshot_is_refreshed+---------------------+t +(1 row)++step s1_commit: COMMIT;++starting permutation: s1_reset_backend s1_check_backend_reset+step s1_reset_backend: + SELECT pg_stat_reset_backend_stats(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++pg_stat_reset_backend_stats+---------------------------+ +(1 row)++step s1_check_backend_reset: + SELECT stats_reset IS NOT NULL AS reset_timestamp_set+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++reset_timestamp_set+-------------------+t +(1 row)+++starting permutation: s1_reset_shared s1_check_shared_reset+step s1_reset_shared: + SELECT pg_stat_reset_shared('wal');++pg_stat_reset_shared+--------------------+ +(1 row)++step s1_check_shared_reset: + SELECT stats_reset IS NOT NULL AS reset_timestamp_set+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));++reset_timestamp_set+-------------------+t +(1 row)+diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_scheduleindex bf9a2037c70..e8444df2956 100644--- a/src/test/isolation/isolation_schedule+++ b/src/test/isolation/isolation_schedule@@ -108,6 +108,7 @@ test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
test: stats
+test: stats-per-backend
test: horizons
test: predicate-hash
test: predicate-gist
diff --git a/src/test/isolation/specs/stats-per-backend.spec b/src/test/isolation/specs/stats-per-backend.specnew file mode 100644index 00000000000..a4343ff8163--- /dev/null+++ b/src/test/isolation/specs/stats-per-backend.spec@@ -0,0 +1,151 @@+# Test snapshot, cache, and reset behavior for per-backend statistics.+#+# Session s2 generates and flushes WAL records, which provide a monotonically+# increasing per-backend counter. WAL is used as a representative statistics+# kind rather than repeating the consistency tests for each kind. Session s1+# verifies that NONE mode always fetches fresh values, SNAPSHOT mode fixes all+# per-backend entries at the initial statistics snapshot, and CACHE mode fixes+# each entry on first access and refreshes it after the transaction ends. The+# final permutations verify reset timestamps after per-backend and shared+# resets.++setup+{+ CREATE TABLE stats_per_backend_data(id int);+ CREATE TABLE stats_per_backend_saved(wal_records bigint);+}++teardown+{+ DROP TABLE stats_per_backend_data;+ DROP TABLE stats_per_backend_saved;+}++session s1+setup { SET stats_fetch_consistency = 'none'; }++step s1_fetch_consistency_none { SET stats_fetch_consistency = 'none'; }+step s1_fetch_consistency_cache { SET stats_fetch_consistency = 'cache'; }+step s1_fetch_consistency_snapshot { SET stats_fetch_consistency = 'snapshot'; }+step s1_begin { BEGIN; }+step s1_commit { COMMIT; }+step s1_build_snapshot {+ SELECT wal_records >= 0 AS snapshot_created FROM pg_stat_wal;+}+step s1_check_live_backend {+ SELECT current.wal_records > saved.wal_records AS snapshot_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;+}+step s1_save_backend_stats {+ INSERT INTO stats_per_backend_saved+ SELECT wal_records+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));+}+step s1_check_cached_backend {+ SELECT current.wal_records = saved.wal_records AS cache_is_stable+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;+}+step s1_check_uncached_backend {+ SELECT current.wal_records > saved.wal_records AS none_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;+}+step s1_check_refreshed_backend {+ SELECT current.wal_records > saved.wal_records AS cache_is_refreshed+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2')) AS current+ CROSS JOIN stats_per_backend_saved AS saved;+}+step s1_reset_backend {+ SELECT pg_stat_reset_backend_stats(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));+}+step s1_check_backend_reset {+ SELECT stats_reset IS NOT NULL AS reset_timestamp_set+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));+}+step s1_reset_shared {+ SELECT pg_stat_reset_shared('wal');+}+step s1_check_shared_reset {+ SELECT stats_reset IS NOT NULL AS reset_timestamp_set+ FROM pg_stat_get_backend_wal(+ (SELECT pid FROM pg_stat_activity+ WHERE application_name = 'isolation/stats-per-backend/s2'));+}++session s2+step s2_generate {+ INSERT INTO stats_per_backend_data VALUES (1);+ SELECT pg_stat_force_next_flush();+}+step s2_generate_more {+ INSERT INTO stats_per_backend_data VALUES (2);+ SELECT pg_stat_force_next_flush();+}++# with stats_fetch_consistency=none s1 should see flushed changes from s2+# despite being in a transaction+permutation+ s1_fetch_consistency_none+ s2_generate+ s1_begin+ s1_save_backend_stats+ s2_generate_more+ s1_check_uncached_backend+ s1_commit++# with stats_fetch_consistency=snapshot s1 should not see flushed changes from+# s2 after building the statistics snapshot, but should see them after commit+permutation+ s1_fetch_consistency_snapshot+ s1_begin+ s1_build_snapshot+ s2_generate+ s1_save_backend_stats+ s1_commit+ s1_check_live_backend++# with stats_fetch_consistency=cache s1 should not see flushed changes from s2+# after the first access, but should see them after commit+permutation+ s1_fetch_consistency_cache+ s2_generate+ s1_begin+ s1_save_backend_stats+ s2_generate_more+ s1_check_cached_backend+ s1_commit+ s1_check_refreshed_backend++# changing from cache to snapshot within a transaction should discard the+# per-backend cache before building the full snapshot+permutation+ s1_fetch_consistency_cache+ s2_generate+ s1_begin+ s1_save_backend_stats+ s2_generate_more+ s1_fetch_consistency_snapshot+ s1_check_live_backend+ s1_commit++# a per-backend reset should set that backend's reset timestamp+permutation s1_reset_backend s1_check_backend_reset++# a shared WAL reset should set the reset timestamp for live backends+permutation s1_reset_shared s1_check_shared_resetdiff --git a/src/test/recovery/t/029_stats_restart.pl b/src/test/recovery/t/029_stats_restart.plindex cdc427dbc78..27cfdb0b75d 100644--- a/src/test/recovery/t/029_stats_restart.pl+++ b/src/test/recovery/t/029_stats_restart.pl@@ -1,7 +1,7 @@
# Copyright (c) 2021-2026, PostgreSQL Global Development Group
-# Tests statistics handling around restarts, including handling of crashes and-# invalid stats files, as well as restoring stats after "normal" restarts.+# Tests statistics snapshot consistency and handling around restarts, including+# crashes, invalid stats files, and restoring stats after normal restarts.
use strict;
use warnings FATAL => 'all';
@@ -18,6 +18,38 @@ $node->start;
my $connect_db = 'postgres';
my $db_under_test = 'test';
+# Verify that a statistics snapshot excludes backends that started after the+# snapshot was created.+my $snapshot_session = $node->background_psql($connect_db);++$snapshot_session->query_safe(+ q[+ BEGIN;+ SET LOCAL stats_fetch_consistency = 'snapshot';+ SELECT count(*) FROM pg_stat_wal;+]);++# Establish this connection only after the first session built its snapshot.+my $late_session = $node->background_psql($connect_db);+my $late_pid = $late_session->query_safe('SELECT pg_backend_pid()');++# WAL returns one scalar composite, so count a field to distinguish a NULL+# result from the fabricated all-zero row.+is( $snapshot_session->query_safe(+ qq[+ SELECT+ (SELECT count(wal_records)+ FROM pg_stat_get_backend_wal($late_pid)),+ (SELECT count(*) FROM pg_stat_get_backend_io($late_pid)),+ (SELECT count(*) FROM pg_stat_get_backend_lock($late_pid));+ ]),+ '0|0|0',+ 'statistics snapshot excludes a backend created later');++$snapshot_session->query_safe('ROLLBACK');+$late_session->quit;+$snapshot_session->quit;+
my $sect = "startup";
# Check some WAL statistics after a fresh startup. The startup process
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.outindex 8b15471248b..e3dd0ce89b0 100644--- a/src/test/regress/expected/stats.out+++ b/src/test/regress/expected/stats.out@@ -1130,6 +1130,35 @@ SELECT wal_bytes > :backend_wal_bytes_before FROM pg_stat_get_backend_wal(pg_bac
t
(1 row)
+-- Test pg_stat_reset_backend_stats() for WAL+SELECT wal_bytes AS wal_bytes_before_reset FROM pg_stat_get_backend_wal(pg_backend_pid()) \gset+SELECT pg_stat_reset_backend_stats(pg_backend_pid());+ pg_stat_reset_backend_stats +-----------------------------+ +(1 row)++SELECT wal_bytes AS wal_bytes_after_reset FROM pg_stat_get_backend_wal(pg_backend_pid()) \gset+SELECT :wal_bytes_after_reset < :wal_bytes_before_reset;+ ?column? +----------+ t+(1 row)++-- global view should still have the data+SELECT wal_bytes >= :wal_bytes_before_reset FROM pg_stat_wal;+ ?column? +----------+ t+(1 row)++-- reset timestamp should be set+SELECT stats_reset IS NOT NULL FROM pg_stat_get_backend_wal(pg_backend_pid());+ ?column? +----------+ t+(1 row)+
-- Test pg_stat_get_backend_idset() and some allied functions.
-- In particular, verify that their notion of backend ID matches
-- our temp schema index.
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sqlindex 674637e172b..f4931fd84de 100644--- a/src/test/regress/sql/stats.sql+++ b/src/test/regress/sql/stats.sql@@ -528,6 +528,16 @@ SELECT wal_bytes > :wal_bytes_before FROM pg_stat_wal;
SELECT pg_stat_force_next_flush();
SELECT wal_bytes > :backend_wal_bytes_before FROM pg_stat_get_backend_wal(pg_backend_pid());
+-- Test pg_stat_reset_backend_stats() for WAL+SELECT wal_bytes AS wal_bytes_before_reset FROM pg_stat_get_backend_wal(pg_backend_pid()) \gset+SELECT pg_stat_reset_backend_stats(pg_backend_pid());+SELECT wal_bytes AS wal_bytes_after_reset FROM pg_stat_get_backend_wal(pg_backend_pid()) \gset+SELECT :wal_bytes_after_reset < :wal_bytes_before_reset;+-- global view should still have the data+SELECT wal_bytes >= :wal_bytes_before_reset FROM pg_stat_wal;+-- reset timestamp should be set+SELECT stats_reset IS NOT NULL FROM pg_stat_get_backend_wal(pg_backend_pid());+
-- Test pg_stat_get_backend_idset() and some allied functions.
-- In particular, verify that their notion of backend ID matches
-- our temp schema index.
--
2.34.1
v4-0002-pgstat-add-new-infrastructure-for-per-backend-sta.patch (text/x-diff, 22.6K) download
From 408726452c655446c9934163edac71af2a64ad93 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 29 Jul 2026 14:11:27 +0000
Subject: [PATCH v4 2/6] pgstat: add new infrastructure for per-backend
statistics
Add infrastructure for per-backend statistics that keeps one shared array entry
per possible process, while retaining the existing shared data as global statistics
for exited processes.
Define a common array entry header containing a BackendType and content
LWLock. B_INVALID marks an unused entry. Extend PgStat_KindInfo with metadata
describing the entry and its statistics payload.
At shared-memory initialization, allocate a ProcNumber-indexed array for every
participating kind. Activate the current process entries during pgstat_initialize(),
so auxiliary and shared-memory-only processes are covered.
Add helpers to fetch per-backend entries, include active entries in global
snapshots, and transfer entries to the global statistics when processes
exit. Reuse the backend-local statistics snapshot hash for per-backend
fetches, keyed by statistics kind and ProcNumber.
No statistics kind registers per-backend metadata in this commit, so the
new shared-memory allocation and process initialization loops are no ops.
Subsequent commits add WAL, lock, and I/O statistics to the infrastructure
individually.
Limit this infrastructure to built-in fixed-numbered statistics kinds, as
this is the only use case so far. It could be extended later if needed.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/meson.build | 1 +
src/backend/utils/activity/pgstat.c | 72 +++-
.../utils/activity/pgstat_per_backend.c | 340 ++++++++++++++++++
src/backend/utils/activity/pgstat_shmem.c | 45 +++
src/include/utils/pgstat_internal.h | 71 ++++
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 529 insertions(+), 2 deletions(-)
81.8% src/backend/utils/activity/
17.9% src/include/utils/
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefileindex 2e32d1485d6..8924b10ed2e 100644--- a/src/backend/utils/activity/Makefile+++ b/src/backend/utils/activity/Makefile@@ -29,6 +29,7 @@ OBJS = \
pgstat_io.o \
pgstat_kind.o \
pgstat_lock.o \
+ pgstat_per_backend.o \
pgstat_relation.o \
pgstat_replslot.o \
pgstat_shmem.o \
diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.buildindex e6dcb2e26fc..6434c377e53 100644--- a/src/backend/utils/activity/meson.build+++ b/src/backend/utils/activity/meson.build@@ -14,6 +14,7 @@ backend_sources += files(
'pgstat_io.c',
'pgstat_kind.c',
'pgstat_lock.c',
+ 'pgstat_per_backend.c',
'pgstat_relation.c',
'pgstat_replslot.c',
'pgstat_shmem.c',
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.cindex d31c0b48a1d..3dd03a71bba 100644--- a/src/backend/utils/activity/pgstat.c+++ b/src/backend/utils/activity/pgstat.c@@ -69,6 +69,7 @@
* To keep things manageable, stats handling is split across several
* files. Infrastructure pieces are in:
* - pgstat.c - this file, to tie it all together
+ * - pgstat_per_backend.c - generic per-backend statistics infrastructure
* - pgstat_shmem.c - nearly everything dealing with shared memory, including
* the maintenance of hashtable entries
* - pgstat_xact.c - transactional integration, including the transactional
@@ -189,7 +190,6 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
-static void pgstat_prep_snapshot(void);
static void pgstat_build_snapshot(void);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
@@ -688,6 +688,13 @@ pgstat_initialize(void)
pgstat_attach_shmem();
+ /*+ * Activate per-backend statistics entries here. This also covers+ * processes that never call InitPostgres(), such as shared-memory-only+ * background workers.+ */+ pgstat_create_my_per_backend_entries();+
pgstat_init_snapshot_fixed();
/* Backend initialization callbacks */
@@ -1155,7 +1162,12 @@ pgstat_init_snapshot_fixed(void)
}
}
-static void+/*+ * Prepare the backend-local statistics snapshot.+ *+ * Clear invalidated snapshot state first. NONE mode does not need a cache.+ */+void
pgstat_prep_snapshot(void)
{
if (force_stats_snapshot_clear)
@@ -1176,6 +1188,57 @@ pgstat_prep_snapshot(void)
NULL);
}
+/*+ * Look up an entry in the backend-local statistics snapshot.+ *+ * The snapshot must have been prepared by pgstat_prep_snapshot(). Set *found+ * for both entries containing data and cached negative lookups.+ */+void *+pgstat_lookup_snapshot_entry(const PgStat_HashKey *key, bool *found)+{+ PgStat_SnapshotEntry *entry;++ Assert(pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE);+ Assert(pgStatLocal.snapshot.stats != NULL);++ entry = pgstat_snapshot_lookup(pgStatLocal.snapshot.stats, *key);+ *found = entry != NULL;++ return entry != NULL ? entry->data : NULL;+}++/*+ * Copy an entry into the backend-local statistics snapshot.+ *+ * A NULL data pointer records a negative lookup. Return an existing entry+ * without replacing it if the key was already cached.+ */+void *+pgstat_cache_snapshot_entry(const PgStat_HashKey *key, const void *data,+ Size data_len)+{+ PgStat_SnapshotEntry *entry;+ bool found;++ Assert(pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE);+ Assert(pgStatLocal.snapshot.stats != NULL);++ entry = pgstat_snapshot_insert(pgStatLocal.snapshot.stats, *key, &found);+ if (found)+ return entry->data;++ if (data != NULL)+ {+ entry->data = MemoryContextAlloc(pgStatLocal.snapshot.context, data_len);+ memcpy(entry->data, data, data_len);+ }+ else+ entry->data = NULL;++ return entry->data;+}+
static void
pgstat_build_snapshot(void)
{
@@ -1560,6 +1623,11 @@ pgstat_register_kind(PgStat_Kind kind, const PgStat_KindInfo *kind_info)
errdetail("Custom cumulative statistics must be registered while initializing modules in \"%s\".",
"shared_preload_libraries")));
+ if (kind_info->per_backend_entry_size != 0)+ ereport(ERROR,+ (errmsg("failed to register custom cumulative statistics \"%s\" with ID %u", kind_info->name, kind),+ errdetail("Per-backend statistics are not supported for custom cumulative statistics.")));+
/*
* Check some data for fixed-numbered stats.
*/
diff --git a/src/backend/utils/activity/pgstat_per_backend.c b/src/backend/utils/activity/pgstat_per_backend.cnew file mode 100644index 00000000000..629559e369f--- /dev/null+++ b/src/backend/utils/activity/pgstat_per_backend.c@@ -0,0 +1,340 @@+/* -------------------------------------------------------------------------+ *+ * pgstat_per_backend.c+ * Generic infrastructure for per-backend statistics.+ *+ * This file manages the per-kind arrays used for per-backend statistics,+ * including entry creation, fetching, snapshots, transfer to global+ * statistics, and invalidation.+ *+ * Copyright (c) 2001-2026, PostgreSQL Global Development Group+ *+ * IDENTIFICATION+ * src/backend/utils/activity/pgstat_per_backend.c+ * -------------------------------------------------------------------------+ */++#include "postgres.h"++#include "storage/proc.h"+#include "utils/pgstat_internal.h"++/*+ * Return one per-backend entry, whether or not it is currently active.+ *+ * Callers that inspect or modify the entry must hold its content lock.+ */+void *+pgstat_get_per_backend_entry(PgStat_Kind kind, ProcNumber procnum)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ char *array;++ if (kind_info == NULL || kind_info->per_backend_entry_size == 0)+ return NULL;++ Assert(pgstat_is_kind_builtin(kind));++ if (procnum < 0 || procnum >= NumPerBackendStatSlots)+ return NULL;++ array = pgStatLocal.shmem->per_backend_arrays[kind];+ Assert(array != NULL);++ return array + procnum * kind_info->per_backend_entry_size;+}++/*+ * Initialize this process's entry for one per-backend statistics kind.+ *+ * A blocking flush during early startup may already have initialized the+ * entry. In that case, preserve the statistics it contains.+ */+static PgStatShared_PerBackendEntry *+pgstat_create_my_per_backend_entry(PgStat_Kind kind)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ PgStatShared_PerBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(kind, MyProcNumber);++ LWLockAcquire(&entry->lock, LW_EXCLUSIVE);++ if (entry->backend_type == B_INVALID)+ {+ entry->backend_type = MyBackendType;+ memset((char *) entry + kind_info->per_backend_data_off, 0,+ kind_info->per_backend_data_len);+ }++ LWLockRelease(&entry->lock);++ return entry;+}++/*+ * Initialize entries for all kinds that use per-backend arrays.+ *+ * This runs after any stale entries for the reused ProcNumber have been+ * transferred to global statistics.+ */+void+pgstat_create_my_per_backend_entries(void)+{+ for (PgStat_Kind kind = PGSTAT_KIND_BUILTIN_MIN;+ kind <= PGSTAT_KIND_BUILTIN_MAX; kind++)+ {+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);++ if (kind_info == NULL || kind_info->per_backend_entry_size == 0)+ continue;++ (void) pgstat_create_my_per_backend_entry(kind);+ }+}++/*+ * Lock this process's entry for a per-backend statistics kind.+ *+ * The blocking path initializes an inactive entry, which allows forced+ * flushes during early process startup. The routine nonblocking path does not+ * initialize entries.+ *+ * NB: On success, the entry is returned with its content lock held. A+ * nonblocking call returns NULL if the entry cannot be locked or is inactive.+ */+void *+pgstat_lock_my_per_backend_entry(PgStat_Kind kind, bool nowait)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ PgStatShared_PerBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(kind, MyProcNumber);++ if (nowait)+ {+ if (!LWLockConditionalAcquire(&entry->lock, LW_EXCLUSIVE))+ return NULL;++ if (entry->backend_type == B_INVALID)+ {+ LWLockRelease(&entry->lock);+ return NULL;+ }+ }+ else+ {+ LWLockAcquire(&entry->lock, LW_EXCLUSIVE);++ if (entry->backend_type == B_INVALID)+ {+ entry->backend_type = MyBackendType;+ memset((char *) entry + kind_info->per_backend_data_off, 0,+ kind_info->per_backend_data_len);+ }+ }++ return entry;+}++/*+ * Accumulate all active per-backend entries into a fixed-kind snapshot.+ *+ * SNAPSHOT mode also copies every active entry into the backend-local+ * snapshot hash, fixing both the values and the set of visible processes.+ * The caller must hold the kind lock to prevent entries from moving into the+ * global statistics while the snapshot is assembled.+ */+void+pgstat_per_backend_snapshot(PgStat_Kind kind, void *snap)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ PgStat_HashKey key = {0};++ key.kind = kind;+ key.dboid = InvalidOid;++ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_PerBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(kind, procnum);++ LWLockAcquire(&entry->lock, LW_SHARED);++ if (entry->backend_type != B_INVALID)+ {+ kind_info->per_backend_acc_cb(snap, entry);++ if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)+ {+ key.objid = procnum;+ pgstat_cache_snapshot_entry(&key,+ (char *) entry ++ kind_info->per_backend_data_off,+ kind_info->per_backend_data_len);+ }+ }++ LWLockRelease(&entry->lock);+ }+}++/*+ * Fetch one process's per-backend statistics.+ *+ * NONE mode returns a copy allocated in the current memory context. CACHE and+ * SNAPSHOT modes return data owned by the backend-local statistics snapshot.+ * Return NULL if the ProcNumber is invalid or no process was active at the+ * consistency mode's observation point.+ */+void *+pgstat_fetch_per_backend(PgStat_Kind kind, ProcNumber procnum)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ PgStatShared_PerBackendEntry *entry;+ PgStat_HashKey key = {0};+ void *stats_data;+ bool found;++ pgstat_prep_snapshot();++ entry = pgstat_get_per_backend_entry(kind, procnum);++ if (entry == NULL)+ return NULL;++ /* Read directly without caching in NONE mode */+ if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_NONE)+ {+ LWLockAcquire(&entry->lock, LW_SHARED);++ if (entry->backend_type == B_INVALID)+ {+ LWLockRelease(&entry->lock);+ return NULL;+ }++ stats_data = palloc(kind_info->per_backend_data_len);+ memcpy(stats_data, (char *) entry + kind_info->per_backend_data_off,+ kind_info->per_backend_data_len);+ LWLockRelease(&entry->lock);++ return stats_data;+ }++ Assert(pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_CACHE ||+ pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT);++ /*+ * A full fixed-kind snapshot pre-caches all active entries. CACHE mode+ * reads only the requested array entry on a cache miss.+ */+ if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)+ pgstat_snapshot_fixed(kind);+ else+ pgStatLocal.snapshot.mode = PGSTAT_FETCH_CONSISTENCY_CACHE;++ key.kind = kind;+ key.dboid = InvalidOid;+ key.objid = procnum;++ stats_data = pgstat_lookup_snapshot_entry(&key, &found);++ if (found)+ return stats_data;++ /*+ * A miss after building a full snapshot means that the process was not+ * active at the snapshot point.+ */+ if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)+ return NULL;++ LWLockAcquire(&entry->lock, LW_SHARED);++ if (entry->backend_type == B_INVALID)+ stats_data = pgstat_cache_snapshot_entry(&key, NULL, 0);+ else+ stats_data = pgstat_cache_snapshot_entry(&key,+ (char *) entry ++ kind_info->per_backend_data_off,+ kind_info->per_backend_data_len);++ LWLockRelease(&entry->lock);++ return stats_data;+}++/*+ * Transfer this process's entry into global statistics and invalidate it.+ *+ * The kind lock serializes this transfer with global snapshots and resets.+ * The entry lock serializes it with routine flushes and per-backend reads.+ */+void+pgstat_acc_my_per_backend(PgStat_Kind kind, LWLock *lock)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ PgStatShared_PerBackendEntry *entry;+ char *shared_struct;+ void *dst;++ entry = pgstat_get_per_backend_entry(kind, MyProcNumber);++ if (entry == NULL)+ return;++ shared_struct = (char *) pgStatLocal.shmem + kind_info->shared_ctl_off;+ dst = shared_struct + kind_info->shared_data_off;++ LWLockAcquire(lock, LW_EXCLUSIVE);+ LWLockAcquire(&entry->lock, LW_EXCLUSIVE);++ if (entry->backend_type != B_INVALID)+ {+ kind_info->per_backend_acc_cb(dst, entry);+ entry->backend_type = B_INVALID;+ }++ LWLockRelease(&entry->lock);+ LWLockRelease(lock);+}++/*+ * Transfer every active entry into global statistics and invalidate it.+ *+ * This is called during clean server shutdown before the statistics file is+ * written.+ */+void+pgstat_acc_all_per_backend(PgStat_Kind kind, LWLock *lock)+{+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ char *shared_struct;+ void *dst;++ shared_struct = (char *) pgStatLocal.shmem + kind_info->shared_ctl_off;+ dst = shared_struct + kind_info->shared_data_off;++ LWLockAcquire(lock, LW_EXCLUSIVE);++ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_PerBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(kind, procnum);++ LWLockAcquire(&entry->lock, LW_EXCLUSIVE);++ if (entry->backend_type != B_INVALID)+ {+ kind_info->per_backend_acc_cb(dst, entry);+ entry->backend_type = B_INVALID;+ }++ LWLockRelease(&entry->lock);+ }++ LWLockRelease(lock);+}diff --git a/src/backend/utils/activity/pgstat_shmem.c b/src/backend/utils/activity/pgstat_shmem.cindex 8500927a68b..c2116fe087f 100644--- a/src/backend/utils/activity/pgstat_shmem.c+++ b/src/backend/utils/activity/pgstat_shmem.c@@ -164,6 +164,21 @@ StatsShmemSize(void)
sz = add_size(sz, MAXALIGN(kind_info->shared_size));
}
+ /* Add the per-backend arrays for built-in fixed-numbered kinds */+ for (PgStat_Kind kind = PGSTAT_KIND_BUILTIN_MIN; kind <= PGSTAT_KIND_BUILTIN_MAX; kind++)+ {+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);++ if (!kind_info || kind_info->per_backend_entry_size == 0)+ continue;++ Assert(kind_info->fixed_amount);+ Assert(kind_info->per_backend_acc_cb != NULL);++ sz = add_size(sz, MAXALIGN(mul_size(NumPerBackendStatSlots,+ kind_info->per_backend_entry_size)));+ }+
return sz;
}
@@ -296,6 +311,36 @@ StatsShmemInit(void *arg)
kind_info->init_shmem_cb(ptr);
}
}
++ /*+ * Allocate and initialize arrays for built-in fixed-numbered kinds that+ * keep per-backend statistics.+ */+ memset(ctl->per_backend_arrays, 0, sizeof(ctl->per_backend_arrays));++ for (PgStat_Kind kind = PGSTAT_KIND_BUILTIN_MIN; kind <= PGSTAT_KIND_BUILTIN_MAX; kind++)+ {+ const PgStat_KindInfo *kind_info = pgstat_get_kind_info(kind);+ Size array_size;++ if (!kind_info || kind_info->per_backend_entry_size == 0)+ continue;++ array_size = mul_size(NumPerBackendStatSlots, kind_info->per_backend_entry_size);++ ctl->per_backend_arrays[kind] = p;++ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_PerBackendEntry *entry;++ entry = (PgStatShared_PerBackendEntry *) (p + procnum * kind_info->per_backend_entry_size);+ LWLockInitialize(&entry->lock, LWTRANCHE_PGSTATS_DATA);+ entry->backend_type = B_INVALID;+ }++ p += MAXALIGN(array_size);+ }
}
void
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.hindex 21403516e28..da5a67cf827 100644--- a/src/include/utils/pgstat_internal.h+++ b/src/include/utils/pgstat_internal.h@@ -19,6 +19,7 @@
#include "lib/ilist.h"
#include "pgstat.h"
#include "storage/lwlock.h"
+#include "storage/proc.h"
#include "utils/dsa.h"
@@ -54,6 +55,8 @@
* NB: We assume that this struct contains no padding. Also, 8 bytes
* allocated for the object ID are good enough to ensure the uniqueness
* of the hash key, hence the addition of new fields is not recommended.
+ * Backend-local snapshots also use this key for per-backend fixed statistics,+ * with InvalidOid as the database ID and ProcNumber as the object ID.
*/
typedef struct PgStat_HashKey
{
@@ -390,6 +393,28 @@ typedef struct PgStat_KindInfo
*/
void (*snapshot_cb) (void);
+ /*+ * Per-backend array metadata for built-in fixed-numbered statistics+ * kinds.+ *+ * A nonzero per_backend_entry_size enables array allocation and process+ * entry initialization. Zero means that the kind has no per-backend+ * entries. The remaining fields describe the kind-specific payload used+ * by the generic fetch, snapshot, and accumulation helpers.+ *+ * Each entry starts with PgStatShared_PerBackendEntry, followed by the+ * kind-specific statistics payload.+ */+ uint32 per_backend_entry_size; /* size of one array entry */+ uint32 per_backend_data_off; /* offset of stats data in entry */+ uint32 per_backend_data_len; /* size of stats data in entry */++ /*+ * Callback to accumulate one per-backend entry into a destination of the+ * kind's statistics type. Called with the entry's content lock held.+ */+ void (*per_backend_acc_cb) (void *dst, void *entry);+
/* name of the kind of stats */
const char *const name;
} PgStat_KindInfo;
@@ -489,6 +514,24 @@ typedef struct PgStatShared_SLRU
PgStat_SLRUStats stats[SLRU_NUM_ELEMENTS];
} PgStatShared_SLRU;
+/*+ * Number of slots in each per-backend statistics array. Prepared+ * transactions do not run code and therefore do not report statistics.+ */+#define NumPerBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)++/*+ * Common header for entries in per-backend statistics arrays.+ *+ * The lock protects the backend type and kind-specific payload that follows+ * this header. B_INVALID marks an unused entry.+ */+typedef struct PgStatShared_PerBackendEntry+{+ LWLock lock;+ BackendType backend_type;+} PgStatShared_PerBackendEntry;+
typedef struct PgStatShared_Wal
{
/* lock protects ->stats */
@@ -599,6 +642,12 @@ typedef struct PgStat_ShmemControl
PgStatShared_SLRU slru;
PgStatShared_Wal wal;
+ /*+ * Per-backend arrays for built-in fixed-numbered statistics kinds.+ * Entries are indexed by ProcNumber.+ */+ void *per_backend_arrays[PGSTAT_KIND_BUILTIN_SIZE];+
/*
* Custom stats data with fixed-numbered objects, indexed by (PgStat_Kind
* - PGSTAT_KIND_CUSTOM_MIN).
@@ -644,6 +693,11 @@ typedef struct PgStat_Snapshot
/* to free snapshot in bulk */
MemoryContext context;
++ /*+ * Cached variable-numbered and per-backend entries, including negative+ * lookups. Entries are owned by the snapshot memory context.+ */
struct pgstat_snapshot_hash *stats;
} PgStat_Snapshot;
@@ -711,6 +765,23 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, uint64 objid,
bool *may_free);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
+extern void pgstat_prep_snapshot(void);+extern void *pgstat_lookup_snapshot_entry(const PgStat_HashKey *key,+ bool *found);+extern void *pgstat_cache_snapshot_entry(const PgStat_HashKey *key,+ const void *data, Size data_len);++/*+ * These functions implement generic per-backend statistics.+ */+extern void *pgstat_get_per_backend_entry(PgStat_Kind kind,+ ProcNumber procnum);+extern void pgstat_create_my_per_backend_entries(void);+extern void *pgstat_lock_my_per_backend_entry(PgStat_Kind kind, bool nowait);+extern void pgstat_per_backend_snapshot(PgStat_Kind kind, void *snap);+extern void *pgstat_fetch_per_backend(PgStat_Kind kind, ProcNumber procnum);+extern void pgstat_acc_my_per_backend(PgStat_Kind kind, LWLock *lock);+extern void pgstat_acc_all_per_backend(PgStat_Kind kind, LWLock *lock);
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.listindex a7a8dbdd23f..1306c96c3e7 100644--- a/src/tools/pgindent/typedefs.list+++ b/src/tools/pgindent/typedefs.list@@ -2302,6 +2302,7 @@ PgStatShared_HashEntry
PgStatShared_Index
PgStatShared_IO
PgStatShared_Lock
+PgStatShared_PerBackendEntry
PgStatShared_Relation
PgStatShared_ReplSlot
PgStatShared_SLRU
--
2.34.1
v4-0003-pgstat-move-WAL-statistics-to-new-per-backend-inf.patch (text/x-diff, 20.7K) download
From c9c28752032900150244110ad4fea87abad57e1f Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Wed, 29 Jul 2026 14:13:28 +0000
Subject: [PATCH v4 3/6] pgstat: move WAL statistics to new per-backend
infrastructure
PGSTAT_KIND_BACKEND stores each backend's WAL, lock, and I/O statistics
together in one variable-numbered entry keyed by ProcNumber. Move WAL
statistics into a dedicated fixed array associated with the WAL statistics
kind and indexed by ProcNumber. The global WAL stats hold data for backends
that have exited, while the array holds statistics for active backends.
Build pg_stat_wal snapshots by copying the global stats and adding every
active per-backend entry.
Transfer the current process's entry into the global stats before invalidating
it at process exit or ProcNumber reuse. Transfer all entries before a clean
shutdown writes the statistics file.
Remove WAL counters from PGSTAT_KIND_BACKEND and use the new infrastructure
in pg_stat_get_backend_wal(). This also makes WAL statistics available for
auxiliary and shared-memory-only workers. Update the documentation to
describe the new behavior.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
doc/src/sgml/monitoring.sgml | 7 +-
src/backend/utils/activity/pgstat.c | 18 +++
src/backend/utils/activity/pgstat_backend.c | 75 ---------
src/backend/utils/activity/pgstat_wal.c | 168 +++++++++++++++++---
src/backend/utils/adt/pgstatfuncs.c | 37 ++++-
src/include/pgstat.h | 3 +-
src/include/utils/pgstat_internal.h | 21 ++-
src/tools/pgindent/typedefs.list | 1 +
8 files changed, 218 insertions(+), 112 deletions(-)
3.6% doc/src/sgml/
77.6% src/backend/utils/activity/
10.3% src/backend/utils/adt/
6.3% src/include/utils/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgmlindex 31ff4a9d99f..f52b859c141 100644--- a/doc/src/sgml/monitoring.sgml+++ b/doc/src/sgml/monitoring.sgml@@ -5868,10 +5868,6 @@ description | Waiting for a newly initialized WAL file to reach durable storage
Returns WAL statistics about the backend with the specified
process ID. The output fields are exactly the same as the ones in the
<structname>pg_stat_wal</structname> view.
- </para>- <para>- The function does not return WAL statistics for the checkpointer,- the background writer, the startup process and the autovacuum launcher.
</para></entry>
</row>
@@ -6047,7 +6043,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage
<listitem>
<para>
<literal>wal</literal>: Reset all the counters shown in the
- <structname>pg_stat_wal</structname> view.+ <structname>pg_stat_wal</structname> view, as well as per-backend+ WAL statistics returned by <function>pg_stat_get_backend_wal</function>.
</para>
</listitem>
<listitem>
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.cindex 3dd03a71bba..9ea09243ccc 100644--- a/src/backend/utils/activity/pgstat.c+++ b/src/backend/utils/activity/pgstat.c@@ -514,6 +514,11 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.init_shmem_cb = pgstat_wal_init_shmem_cb,
.reset_all_cb = pgstat_wal_reset_all_cb,
.snapshot_cb = pgstat_wal_snapshot_cb,
++ .per_backend_entry_size = sizeof(PgStatShared_WalBackendEntry),+ .per_backend_data_off = offsetof(PgStatShared_WalBackendEntry, stats),+ .per_backend_data_len = sizeof(PgStat_WalStats),+ .per_backend_acc_cb = pgstat_wal_per_backend_acc_cb,
},
};
@@ -625,6 +630,9 @@ pgstat_before_server_shutdown(int code, Datum arg)
*/
if (code == 0)
{
+ /* Transfer all live per-backend stats before writing the stats file */+ pgstat_wal_acc_all_backends();+
pgStatLocal.shmem->is_shutdown = true;
pgstat_write_statsfile();
}
@@ -668,6 +676,9 @@ pgstat_shutdown_hook(int code, Datum arg)
if (!pgstat_drop_entry(PGSTAT_KIND_BACKEND, InvalidOid, MyProcNumber, false))
pgstat_request_entry_refs_gc();
+ /* Accumulate per-backend WAL stats into the global stats */+ pgstat_wal_acc_backend_cb();+
pgstat_detach_shmem();
#ifdef USE_ASSERT_CHECKING
@@ -688,6 +699,13 @@ pgstat_initialize(void)
pgstat_attach_shmem();
+ /*+ * A previous process may have left statistics in entries for this reused+ * ProcNumber. Transfer them to the global statistics before activating+ * the current process's entries.+ */+ pgstat_wal_acc_backend_cb();+
/*
* Activate per-backend statistics entries here. This also covers
* processes that never call InitPostgres(), such as shared-memory-only
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.cindex b736b2ccc6f..76b970a8c41 100644--- a/src/backend/utils/activity/pgstat_backend.c+++ b/src/backend/utils/activity/pgstat_backend.c@@ -41,14 +41,6 @@ static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
-/*- * WAL usage counters saved from pgWalUsage at the previous call to- * pgstat_flush_backend(). This is used to calculate how much WAL usage- * happens between pgstat_flush_backend() calls, by subtracting the- * previous counters from the current ones.- */-static WalUsage prevBackendWalUsage;-
/*
* Utility routines to report I/O stats for backends, kept here to avoid
* exposing PendingBackendStats to the outside world.
@@ -244,58 +236,6 @@ pgstat_flush_backend_entry_io(PgStat_EntryRef *entry_ref)
backend_has_iostats = false;
}
-/*- * To determine whether WAL usage happened.- */-static inline bool-pgstat_backend_wal_have_pending(void)-{- return (pgWalUsage.wal_records != prevBackendWalUsage.wal_records);-}--/*- * Flush out locally pending backend WAL statistics. Locking is managed- * by the caller.- */-static void-pgstat_flush_backend_entry_wal(PgStat_EntryRef *entry_ref)-{- PgStatShared_Backend *shbackendent;- PgStat_WalCounters *bktype_shstats;- WalUsage wal_usage_diff = {0};-- /*- * This function can be called even if nothing at all has happened for WAL- * statistics. In this case, avoid unnecessarily modifying the stats- * entry.- */- if (!pgstat_backend_wal_have_pending())- return;-- shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;- bktype_shstats = &shbackendent->stats.wal_counters;-- /*- * Calculate how much WAL usage counters were increased by subtracting the- * previous counters from the current ones.- */- WalUsageAccumDiff(&wal_usage_diff, &pgWalUsage, &prevBackendWalUsage);--#define WALSTAT_ACC(fld, var_to_add) \- (bktype_shstats->fld += var_to_add.fld)- WALSTAT_ACC(wal_buffers_full, wal_usage_diff);- WALSTAT_ACC(wal_records, wal_usage_diff);- WALSTAT_ACC(wal_fpi, wal_usage_diff);- WALSTAT_ACC(wal_bytes, wal_usage_diff);- WALSTAT_ACC(wal_fpi_bytes, wal_usage_diff);-#undef WALSTAT_ACC-- /*- * Save the current counters for the subsequent calculation of WAL usage.- */- prevBackendWalUsage = pgWalUsage;-}-
/*
* Flush out locally pending backend lock statistics. Locking is managed
* by the caller.
@@ -345,11 +285,6 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_IO) && backend_has_iostats)
has_pending_data = true;
- /* Some WAL data pending? */- if ((flags & PGSTAT_BACKEND_FLUSH_WAL) &&- pgstat_backend_wal_have_pending())- has_pending_data = true;-
/* Some lock data pending? */
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
@@ -366,9 +301,6 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_IO)
pgstat_flush_backend_entry_io(entry_ref);
- if (flags & PGSTAT_BACKEND_FLUSH_WAL)- pgstat_flush_backend_entry_wal(entry_ref);-
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
@@ -411,13 +343,6 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
-- /*- * Initialize prevBackendWalUsage with pgWalUsage so that- * pgstat_backend_flush_cb() can calculate how much pgWalUsage counters- * are increased by subtracting prevBackendWalUsage from pgWalUsage.- */- prevBackendWalUsage = pgWalUsage;
}
/*
diff --git a/src/backend/utils/activity/pgstat_wal.c b/src/backend/utils/activity/pgstat_wal.cindex 183e0a7a97b..8009ac38801 100644--- a/src/backend/utils/activity/pgstat_wal.c+++ b/src/backend/utils/activity/pgstat_wal.c@@ -8,6 +8,11 @@
* storage implementation and the details about individual types of
* statistics.
*
+ * WAL statistics use a per-backend array to avoid double-counting. Each+ * backend flushes WAL usage to its own ProcNumber-indexed entry. The global+ * pg_stat_wal view aggregates the global stats (which holds stats from exited+ * backends) plus all active per-backend entries.+ *
* Copyright (c) 2001-2026, PostgreSQL Global Development Group
*
* IDENTIFICATION
@@ -52,7 +57,6 @@ pgstat_report_wal(bool force)
/* flush wal stats */
(void) pgstat_wal_flush_cb(nowait);
- pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_WAL);
/* flush IO stats */
pgstat_flush_io(nowait);
@@ -81,8 +85,8 @@ pgstat_wal_have_pending(void)
}
/*
- * Calculate how much WAL usage counters have increased by subtracting the- * previous counters from the current ones.+ * Flush WAL usage accumulated since the previous call to this process's+ * per-backend entry.
*
* If nowait is true, this function returns true if the lock could not be
* acquired. Otherwise return false.
@@ -90,7 +94,7 @@ pgstat_wal_have_pending(void)
bool
pgstat_wal_flush_cb(bool nowait)
{
- PgStatShared_Wal *stats_shmem = &pgStatLocal.shmem->wal;+ PgStatShared_WalBackendEntry *entry;
WalUsage wal_usage_diff = {0};
Assert(IsUnderPostmaster || !IsPostmasterEnvironment);
@@ -105,19 +109,18 @@ pgstat_wal_flush_cb(bool nowait)
return false;
/*
- * We don't update the WAL usage portion of the local WalStats elsewhere.
* Calculate how much WAL usage counters were increased by subtracting the
* previous counters from the current ones.
*/
WalUsageAccumDiff(&wal_usage_diff, &pgWalUsage, &prevWalUsage);
- if (!nowait)- LWLockAcquire(&stats_shmem->lock, LW_EXCLUSIVE);- else if (!LWLockConditionalAcquire(&stats_shmem->lock, LW_EXCLUSIVE))- return true;+ entry = pgstat_lock_my_per_backend_entry(PGSTAT_KIND_WAL, nowait);++ if (entry == NULL)+ return nowait;
#define WALSTAT_ACC(fld, var_to_add) \
- (stats_shmem->stats.wal_counters.fld += var_to_add.fld)+ (entry->stats.wal_counters.fld += var_to_add.fld)
WALSTAT_ACC(wal_records, wal_usage_diff);
WALSTAT_ACC(wal_fpi, wal_usage_diff);
WALSTAT_ACC(wal_bytes, wal_usage_diff);
@@ -125,7 +128,7 @@ pgstat_wal_flush_cb(bool nowait)
WALSTAT_ACC(wal_buffers_full, wal_usage_diff);
#undef WALSTAT_ACC
- LWLockRelease(&stats_shmem->lock);+ LWLockRelease(&entry->header.lock);
/*
* Save the current counters for the subsequent calculation of WAL usage.
@@ -157,21 +160,144 @@ pgstat_wal_init_shmem_cb(void *stats)
void
pgstat_wal_reset_all_cb(TimestampTz ts)
{
- PgStatShared_Wal *stats_shmem = &pgStatLocal.shmem->wal;+ PgStatShared_Wal *shmem = &pgStatLocal.shmem->wal;++ /*+ * Hold the kind lock while resetting both the global stats and active+ * entries. Transfers hold the same lock, so pre-reset counters cannot be+ * moved into the global stats after it is reset.+ */+ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ memset(&shmem->stats, 0, sizeof(shmem->stats));+ shmem->stats.stat_reset_timestamp = ts;++ /* Reset all per-backend entries */+ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_WalBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_WAL, procnum);- LWLockAcquire(&stats_shmem->lock, LW_EXCLUSIVE);- memset(&stats_shmem->stats, 0, sizeof(stats_shmem->stats));- stats_shmem->stats.stat_reset_timestamp = ts;- LWLockRelease(&stats_shmem->lock);+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);+ if (entry->header.backend_type != B_INVALID)+ {+ memset(&entry->stats.wal_counters, 0, sizeof(PgStat_WalCounters));+ entry->stats.stat_reset_timestamp = ts;+ }+ LWLockRelease(&entry->header.lock);+ }++ LWLockRelease(&shmem->lock);
}
+/*+ * Build WAL stats snapshot by aggregating global stats and all live+ * per-backend entries.+ */
void
pgstat_wal_snapshot_cb(void)
{
- PgStatShared_Wal *stats_shmem = &pgStatLocal.shmem->wal;+ PgStatShared_Wal *shmem = &pgStatLocal.shmem->wal;+ PgStat_WalStats *snap = &pgStatLocal.snapshot.wal;++ /*+ * Prevent entries from moving to the global stats between copying it and+ * scanning the per-backend array.+ */+ LWLockAcquire(&shmem->lock, LW_SHARED);+ memcpy(snap, &shmem->stats, sizeof(PgStat_WalStats));++ /* Add in all active per-backend entries */+ pgstat_per_backend_snapshot(PGSTAT_KIND_WAL, snap);- LWLockAcquire(&stats_shmem->lock, LW_SHARED);- memcpy(&pgStatLocal.snapshot.wal, &stats_shmem->stats,- sizeof(pgStatLocal.snapshot.wal));- LWLockRelease(&stats_shmem->lock);+ LWLockRelease(&shmem->lock);+}++#define WAL_ACCUMULATE_COUNTERS(dst, src) \+do { \+ (dst).wal_records += (src).wal_records; \+ (dst).wal_fpi += (src).wal_fpi; \+ (dst).wal_bytes += (src).wal_bytes; \+ (dst).wal_fpi_bytes += (src).wal_fpi_bytes; \+ (dst).wal_buffers_full += (src).wal_buffers_full; \+} while (0)++/*+ * Accumulate one per-backend WAL entry into a snapshot or the global stats.+ */+void+pgstat_wal_per_backend_acc_cb(void *dst, void *entry)+{+ PgStat_WalStats *stats = dst;+ PgStatShared_WalBackendEntry *e = (PgStatShared_WalBackendEntry *) entry;++ WAL_ACCUMULATE_COUNTERS(stats->wal_counters, e->stats.wal_counters);+}++/*+ * Accumulate a backend's WAL stats into the global stats, then invalidate the+ * entry.+ *+ * Called at backend exit after the final flush, or when a ProcNumber is+ * being reused.+ */+void+pgstat_wal_acc_backend_cb(void)+{+ pgstat_acc_my_per_backend(PGSTAT_KIND_WAL, &pgStatLocal.shmem->wal.lock);+}++/*+ * Returns per-backend WAL statistics for the given ProcNumber.+ */+PgStat_WalStats *+pgstat_fetch_stat_backend_wal(ProcNumber procnum)+{+ return (PgStat_WalStats *) pgstat_fetch_per_backend(PGSTAT_KIND_WAL, procnum);+}++/*+ * Reset a backend's WAL stats. Accumulate the entry's counters into the+ * global stats, then zero the stats and set the reset timestamp.+ */+void+pgstat_wal_reset_backend_cb(ProcNumber procnum, TimestampTz ts)+{+ PgStatShared_Wal *shmem = &pgStatLocal.shmem->wal;+ PgStatShared_WalBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_WAL, procnum);+ if (entry == NULL)+ return;++ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);++ if (entry->header.backend_type == B_INVALID)+ {+ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);+ return;+ }++ /* Accumulate current stats into global before zeroing */+ WAL_ACCUMULATE_COUNTERS(shmem->stats.wal_counters, entry->stats.wal_counters);++ /* Zero stats and set reset timestamp */+ memset(&entry->stats.wal_counters, 0, sizeof(PgStat_WalCounters));+ entry->stats.stat_reset_timestamp = ts;++ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);+}++/*+ * Accumulate all per-backend WAL stats entries into the global stats and+ * invalidate them. Called at clean server shutdown to ensure all flushed data+ * is preserved in the stats file.+ */+void+pgstat_wal_acc_all_backends(void)+{+ pgstat_acc_all_per_backend(PGSTAT_KIND_WAL, &pgStatLocal.shmem->wal.lock);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.cindex 64b6f60516c..ef541205db4 100644--- a/src/backend/utils/adt/pgstatfuncs.c+++ b/src/backend/utils/adt/pgstatfuncs.c@@ -1769,19 +1769,33 @@ Datum
pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
{
int pid;
- PgStat_Backend *backend_stats;- PgStat_WalCounters bktype_stats;+ PGPROC *proc;+ ProcNumber procnum;+ PgBackendStatus *beentry;+ PgStat_WalStats *wal_stats;
pid = PG_GETARG_INT32(0);
- backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);- if (!backend_stats)+ proc = BackendPidGetProc(pid);++ if (!proc)+ proc = AuxiliaryPidGetProc(pid);+ if (!proc)+ PG_RETURN_NULL();++ procnum = GetNumberFromPGProc(proc);+ beentry = pgstat_get_beentry_by_proc_number(procnum);++ if (!beentry || beentry->st_procpid != pid)
PG_RETURN_NULL();
- bktype_stats = backend_stats->wal_counters;+ wal_stats = pgstat_fetch_stat_backend_wal(procnum);- /* save tuples with data from this PgStat_WalCounters */- return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));+ if (!wal_stats)+ PG_RETURN_NULL();++ return (pg_stat_wal_build_tuple(wal_stats->wal_counters,+ wal_stats->stat_reset_timestamp));
}
/*
@@ -2175,6 +2189,7 @@ pg_stat_reset_backend_stats(PG_FUNCTION_ARGS)
PGPROC *proc;
PgBackendStatus *beentry;
ProcNumber procNumber;
+ TimestampTz ts;
int backend_pid = PG_GETARG_INT32(0);
proc = BackendPidGetProc(backend_pid);
@@ -2196,6 +2211,14 @@ pg_stat_reset_backend_stats(PG_FUNCTION_ARGS)
if (!pgstat_tracks_backend_bktype(beentry->st_backendType))
PG_RETURN_VOID();
+ /*+ * Accumulate the backend's WAL stats into the global stats, then zero the+ * entry.+ */+ ts = GetCurrentTimestamp();+ pgstat_wal_reset_backend_cb(procNumber, ts);++ /* Reset IO and Lock stats still in PGSTAT_KIND_BACKEND */
pgstat_reset(PGSTAT_KIND_BACKEND, InvalidOid, procNumber);
PG_RETURN_VOID();
diff --git a/src/include/pgstat.h b/src/include/pgstat.hindex 187d82c96fe..9a71b8d3cd8 100644--- a/src/include/pgstat.h+++ b/src/include/pgstat.h@@ -593,7 +593,6 @@ typedef struct PgStat_Backend
{
TimestampTz stat_reset_timestamp;
PgStat_BktypeIO io_stats;
- PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
} PgStat_Backend;
@@ -946,6 +945,8 @@ extern void pgstat_execute_transactional_drops(int ndrops, struct xl_xact_stats_
extern void pgstat_report_wal(bool force);
extern PgStat_WalStats *pgstat_fetch_stat_wal(void);
+extern PgStat_WalStats *pgstat_fetch_stat_backend_wal(ProcNumber procnum);+extern void pgstat_wal_reset_backend_cb(ProcNumber procnum, TimestampTz ts);
/*
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.hindex da5a67cf827..d0c9379b42b 100644--- a/src/include/utils/pgstat_internal.h+++ b/src/include/utils/pgstat_internal.h@@ -532,9 +532,22 @@ typedef struct PgStatShared_PerBackendEntry
BackendType backend_type;
} PgStatShared_PerBackendEntry;
+/*+ * Per-backend entry for WAL statistics, stored in an array indexed by+ * ProcNumber.+ */+typedef struct PgStatShared_WalBackendEntry+{+ PgStatShared_PerBackendEntry header;+ PgStat_WalStats stats;+} PgStatShared_WalBackendEntry;+
typedef struct PgStatShared_Wal
{
- /* lock protects ->stats */+ /*+ * The lock protects global stats and serializes snapshots and resets with+ * transfers from per-backend entries.+ */
LWLock lock;
PgStat_WalStats stats;
} PgStatShared_Wal;
@@ -798,9 +811,8 @@ extern void pgstat_archiver_snapshot_cb(void);
/* flags for pgstat_flush_backend() */
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
-#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_LOCK)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
@@ -950,6 +962,9 @@ extern bool pgstat_wal_flush_cb(bool nowait);
extern void pgstat_wal_init_shmem_cb(void *stats);
extern void pgstat_wal_reset_all_cb(TimestampTz ts);
extern void pgstat_wal_snapshot_cb(void);
+extern void pgstat_wal_acc_backend_cb(void);+extern void pgstat_wal_acc_all_backends(void);+extern void pgstat_wal_per_backend_acc_cb(void *dst, void *entry);
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.listindex 1306c96c3e7..95b87d32a62 100644--- a/src/tools/pgindent/typedefs.list+++ b/src/tools/pgindent/typedefs.list@@ -2308,6 +2308,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStatShared_WalBackendEntry
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
v4-0004-pgstat-move-lock-statistics-to-new-per-backend-in.patch (text/x-diff, 21.1K) download
From 894e6a8781423da21b61875f65ddd29e0de71c17 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Fri, 17 Jul 2026 13:25:10 +0000
Subject: [PATCH v4 4/6] pgstat: move lock statistics to new per-backend
infrastructure
PGSTAT_KIND_BACKEND stores each backend's lock and I/O statistics together
in one variable-numbered entry keyed by ProcNumber. Move lock statistics
into a dedicated fixed array associated with the lock statistics kind and
indexed by ProcNumber. The global lock stats hold data for backends that
have exited, while the array holds statistics for active backends.
Build pg_stat_lock snapshots by copying the global stats and adding every
active per-backend entry.
Transfer the current process's entry into the global stats before
invalidating it at process exit or ProcNumber reuse. Transfer all entries
before a clean shutdown writes the statistics file.
Remove lock counters from PGSTAT_KIND_BACKEND and use the new infrastructure
in pg_stat_get_backend_lock(). This also makes lock statistics available for
auxiliary and shared-memory-only workers. Update the documentation to
describe the new behavior.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
doc/src/sgml/monitoring.sgml | 8 +-
src/backend/utils/activity/pgstat.c | 10 +-
src/backend/utils/activity/pgstat_backend.c | 72 ---------
src/backend/utils/activity/pgstat_lock.c | 170 +++++++++++++++++---
src/backend/utils/adt/pgstatfuncs.c | 34 +++-
src/include/pgstat.h | 12 +-
src/include/utils/pgstat_internal.h | 21 ++-
src/tools/pgindent/typedefs.list | 1 +
8 files changed, 205 insertions(+), 123 deletions(-)
3.7% doc/src/sgml/
73.9% src/backend/utils/activity/
10.1% src/backend/utils/adt/
7.0% src/include/utils/
4.8% src/include/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgmlindex f52b859c141..0602d80e599 100644--- a/doc/src/sgml/monitoring.sgml+++ b/doc/src/sgml/monitoring.sgml@@ -5849,10 +5849,6 @@ description | Waiting for a newly initialized WAL file to reach durable storage
Returns lock statistics about the backend with the specified
process ID. The output fields are exactly the same as the ones in the
<structname>pg_stat_lock</structname> view.
- </para>- <para>- The function does not return lock statistics for the checkpointer,- the background writer, the startup process and the autovacuum launcher.
</para></entry>
</row>
@@ -6025,7 +6021,9 @@ description | Waiting for a newly initialized WAL file to reach durable storage
<listitem>
<para>
<literal>lock</literal>: Reset all the counters shown in the
- <structname>pg_stat_lock</structname> view.+ <structname>pg_stat_lock</structname> view, as well as per-backend+ lock statistics returned by+ <function>pg_stat_get_backend_lock</function>.
</para>
</listitem>
<listitem>
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.cindex 9ea09243ccc..1affde4df05 100644--- a/src/backend/utils/activity/pgstat.c+++ b/src/backend/utils/activity/pgstat.c@@ -479,6 +479,11 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.init_shmem_cb = pgstat_lock_init_shmem_cb,
.reset_all_cb = pgstat_lock_reset_all_cb,
.snapshot_cb = pgstat_lock_snapshot_cb,
++ .per_backend_entry_size = sizeof(PgStatShared_LockBackendEntry),+ .per_backend_data_off = offsetof(PgStatShared_LockBackendEntry, stats),+ .per_backend_data_len = sizeof(PgStat_Lock),+ .per_backend_acc_cb = pgstat_lock_per_backend_acc_cb,
},
[PGSTAT_KIND_SLRU] = {
@@ -632,6 +637,7 @@ pgstat_before_server_shutdown(int code, Datum arg)
{
/* Transfer all live per-backend stats before writing the stats file */
pgstat_wal_acc_all_backends();
+ pgstat_lock_acc_all_backends();
pgStatLocal.shmem->is_shutdown = true;
pgstat_write_statsfile();
@@ -676,8 +682,9 @@ pgstat_shutdown_hook(int code, Datum arg)
if (!pgstat_drop_entry(PGSTAT_KIND_BACKEND, InvalidOid, MyProcNumber, false))
pgstat_request_entry_refs_gc();
- /* Accumulate per-backend WAL stats into the global stats */+ /* Accumulate per-backend stats into the global stats */
pgstat_wal_acc_backend_cb();
+ pgstat_lock_acc_backend_cb();
pgstat_detach_shmem();
@@ -705,6 +712,7 @@ pgstat_initialize(void)
* the current process's entries.
*/
pgstat_wal_acc_backend_cb();
+ pgstat_lock_acc_backend_cb();
/*
* Activate per-backend statistics entries here. This also covers
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.cindex 76b970a8c41..8dda2cd88c1 100644--- a/src/backend/utils/activity/pgstat_backend.c+++ b/src/backend/utils/activity/pgstat_backend.c@@ -39,7 +39,6 @@
*/
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
-static bool backend_has_lockstats = false;
/*
* Utility routines to report I/O stats for backends, kept here to avoid
@@ -79,39 +78,6 @@ pgstat_count_backend_io_op(IOObject io_object, IOContext io_context,
pgstat_report_fixed = true;
}
-/*- * Utility routines to report lock stats for backends, kept here to avoid- * exposing PendingBackendStats to the outside world.- */-void-pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs)-{- if (!pgstat_tracks_backend_bktype(MyBackendType))- return;-- Assert(locktag_type <= LOCKTAG_LAST_TYPE);-- PendingBackendStats.pending_lock.stats[locktag_type].waits++;- PendingBackendStats.pending_lock.stats[locktag_type].wait_time += usecs;-- backend_has_lockstats = true;- pgstat_report_fixed = true;-}--void-pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)-{- if (!pgstat_tracks_backend_bktype(MyBackendType))- return;-- Assert(locktag_type <= LOCKTAG_LAST_TYPE);-- PendingBackendStats.pending_lock.stats[locktag_type].fastpath_exceeded++;-- backend_has_lockstats = true;- pgstat_report_fixed = true;-}-
/*
* Returns statistics of a backend by proc number.
*/
@@ -236,36 +202,6 @@ pgstat_flush_backend_entry_io(PgStat_EntryRef *entry_ref)
backend_has_iostats = false;
}
-/*- * Flush out locally pending backend lock statistics. Locking is managed- * by the caller.- */-static void-pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)-{- PgStatShared_Backend *shbackendent;- PgStat_PendingLock *bktype_shstats;-- if (!backend_has_lockstats)- return;-- shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;- bktype_shstats = &shbackendent->stats.lock_stats;-- for (int i = 0; i <= LOCKTAG_LAST_TYPE; i++)- {-#define LOCKSTAT_ACC(fld) \- (bktype_shstats->stats[i].fld += PendingBackendStats.pending_lock.stats[i].fld)- LOCKSTAT_ACC(waits);- LOCKSTAT_ACC(wait_time);- LOCKSTAT_ACC(fastpath_exceeded);-#undef LOCKSTAT_ACC- }-- MemSet(&PendingBackendStats.pending_lock, 0, sizeof(PgStat_PendingLock));- backend_has_lockstats = false;-}-
/*
* Flush out locally pending backend statistics
*
@@ -285,10 +221,6 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_IO) && backend_has_iostats)
has_pending_data = true;
- /* Some lock data pending? */- if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)- has_pending_data = true;-
if (!has_pending_data)
return false;
@@ -301,9 +233,6 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_IO)
pgstat_flush_backend_entry_io(entry_ref);
- if (flags & PGSTAT_BACKEND_FLUSH_LOCK)- pgstat_flush_backend_entry_lock(entry_ref);-
pgstat_unlock_entry(entry_ref);
return false;
@@ -342,7 +271,6 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
- backend_has_lockstats = false;
}
/*
diff --git a/src/backend/utils/activity/pgstat_lock.c b/src/backend/utils/activity/pgstat_lock.cindex c20c7599683..ab226dfd893 100644--- a/src/backend/utils/activity/pgstat_lock.c+++ b/src/backend/utils/activity/pgstat_lock.c@@ -8,6 +8,12 @@
* access / storage implementation and the details about individual types
* of statistics.
*
+ * Lock statistics use a per-backend array to avoid double-counting. Each+ * backend flushes lock stats (waits, wait_time, fastpath_exceeded per lock+ * tag type) to its own ProcNumber-indexed entry. The global pg_stat_lock view+ * aggregates the global stats (which holds stats from exited backends) plus+ * all active per-backend entries.+ *
* Copyright (c) 2021-2026, PostgreSQL Global Development Group
*
* IDENTIFICATION
@@ -40,7 +46,7 @@ pgstat_lock_flush(bool nowait)
}
/*
- * Flush out locally pending lock statistics+ * Flush out locally pending lock statistics to the per-backend array entry.
*
* If no stats have been recorded, this function returns false.
*
@@ -50,31 +56,27 @@ pgstat_lock_flush(bool nowait)
bool
pgstat_lock_flush_cb(bool nowait)
{
- LWLock *lckstat_lock;- PgStatShared_Lock *shstats;+ PgStatShared_LockBackendEntry *entry;
if (!have_lockstats)
return false;
- shstats = &pgStatLocal.shmem->lock;- lckstat_lock = &shstats->lock;+ entry = pgstat_lock_my_per_backend_entry(PGSTAT_KIND_LOCK, nowait);- if (!nowait)- LWLockAcquire(lckstat_lock, LW_EXCLUSIVE);- else if (!LWLockConditionalAcquire(lckstat_lock, LW_EXCLUSIVE))- return true;+ if (entry == NULL)+ return nowait;
for (int i = 0; i <= LOCKTAG_LAST_TYPE; i++)
{
#define LOCKSTAT_ACC(fld) \
- (shstats->stats.stats[i].fld += PendingLockStats.stats[i].fld)+ (entry->stats.stats[i].fld += PendingLockStats.stats[i].fld)
LOCKSTAT_ACC(waits);
LOCKSTAT_ACC(wait_time);
LOCKSTAT_ACC(fastpath_exceeded);
#undef LOCKSTAT_ACC
}
- LWLockRelease(lckstat_lock);+ LWLockRelease(&entry->header.lock);
memset(&PendingLockStats, 0, sizeof(PendingLockStats));
have_lockstats = false;
@@ -93,28 +95,117 @@ pgstat_lock_init_shmem_cb(void *stats)
void
pgstat_lock_reset_all_cb(TimestampTz ts)
{
- LWLock *lckstat_lock = &pgStatLocal.shmem->lock.lock;+ PgStatShared_Lock *shmem = &pgStatLocal.shmem->lock;++ /*+ * Hold the kind lock while resetting both the global stats and active+ * entries. Transfers hold the same lock, so pre-reset counters cannot be+ * moved into the global stats after it is reset.+ */+ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ memset(&shmem->stats, 0, sizeof(shmem->stats));+ shmem->stats.stat_reset_timestamp = ts;- LWLockAcquire(lckstat_lock, LW_EXCLUSIVE);+ /* Reset all per-backend entries, since they contribute to the global view */+ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_LockBackendEntry *entry;- pgStatLocal.shmem->lock.stats.stat_reset_timestamp = ts;+ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_LOCK, procnum);- memset(pgStatLocal.shmem->lock.stats.stats, 0,- sizeof(pgStatLocal.shmem->lock.stats.stats));+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);+ if (entry->header.backend_type != B_INVALID)+ {+ memset(&entry->stats.stats, 0, sizeof(entry->stats.stats));+ entry->stats.stat_reset_timestamp = ts;+ }+ LWLockRelease(&entry->header.lock);+ }- LWLockRelease(lckstat_lock);+ LWLockRelease(&shmem->lock);
}
+/*+ * Build lock stats snapshot by aggregating global stats and all active+ * per-backend entries.+ */
void
pgstat_lock_snapshot_cb(void)
{
- LWLock *lckstat_lock = &pgStatLocal.shmem->lock.lock;+ PgStatShared_Lock *shmem = &pgStatLocal.shmem->lock;+ PgStat_Lock *snap = &pgStatLocal.snapshot.lock;- LWLockAcquire(lckstat_lock, LW_SHARED);+ /*+ * Prevent entries from moving to the global stats between copying it and+ * scanning the per-backend array.+ */+ LWLockAcquire(&shmem->lock, LW_SHARED);+ memcpy(snap, &shmem->stats, sizeof(PgStat_Lock));- pgStatLocal.snapshot.lock = pgStatLocal.shmem->lock.stats;+ /* Add in all active per-backend entries */+ pgstat_per_backend_snapshot(PGSTAT_KIND_LOCK, snap);- LWLockRelease(lckstat_lock);+ LWLockRelease(&shmem->lock);+}++/*+ * Accumulate one per-backend lock entry into a snapshot or the global stats.+ */+void+pgstat_lock_per_backend_acc_cb(void *dst, void *entry)+{+ PgStat_Lock *stats = dst;+ PgStatShared_LockBackendEntry *e = (PgStatShared_LockBackendEntry *) entry;++ for (int j = 0; j <= LOCKTAG_LAST_TYPE; j++)+ {+ stats->stats[j].waits += e->stats.stats[j].waits;+ stats->stats[j].wait_time += e->stats.stats[j].wait_time;+ stats->stats[j].fastpath_exceeded += e->stats.stats[j].fastpath_exceeded;+ }+}++#define LOCK_ACCUMULATE_COUNTERS(dst, src) \+do { \+ for (int _i = 0; _i <= LOCKTAG_LAST_TYPE; _i++) \+ { \+ (dst)[_i].waits += (src)[_i].waits; \+ (dst)[_i].wait_time += (src)[_i].wait_time; \+ (dst)[_i].fastpath_exceeded += (src)[_i].fastpath_exceeded; \+ } \+} while (0)++/*+ * Accumulate a backend's lock stats into the global stats, then invalidate the+ * entry.+ *+ * Called at backend exit after the final flush, or when a ProcNumber is+ * being reused.+ */+void+pgstat_lock_acc_backend_cb(void)+{+ pgstat_acc_my_per_backend(PGSTAT_KIND_LOCK, &pgStatLocal.shmem->lock.lock);+}++/*+ * Accumulate all per-backend lock stats entries into the global stats and+ * invalidate them. Called at clean server shutdown to ensure all flushed data+ * is preserved in the stats file.+ */+void+pgstat_lock_acc_all_backends(void)+{+ pgstat_acc_all_per_backend(PGSTAT_KIND_LOCK, &pgStatLocal.shmem->lock.lock);+}++/*+ * Returns per-backend lock statistics for the given ProcNumber.+ */+PgStat_Lock *+pgstat_fetch_stat_backend_lock(ProcNumber procnum)+{+ return (PgStat_Lock *) pgstat_fetch_per_backend(PGSTAT_KIND_LOCK, procnum);
}
/*
@@ -131,8 +222,6 @@ pgstat_count_lock_fastpath_exceeded(uint8 locktag_type)
PendingLockStats.stats[locktag_type].fastpath_exceeded++;
have_lockstats = true;
pgstat_report_fixed = true;
-- pgstat_count_backend_lock_fastpath_exceeded(locktag_type);
}
/*
@@ -149,6 +238,39 @@ pgstat_count_lock_waits(uint8 locktag_type, PgStat_Counter usecs)
PendingLockStats.stats[locktag_type].wait_time += usecs;
have_lockstats = true;
pgstat_report_fixed = true;
+}++/*+ * Reset a backend's lock stats. Accumulate the entry's counters into the+ * global stats, then zero the stats and set the reset timestamp.+ */+void+pgstat_lock_reset_backend_cb(ProcNumber procnum, TimestampTz ts)+{+ PgStatShared_Lock *shmem = &pgStatLocal.shmem->lock;+ PgStatShared_LockBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_LOCK, procnum);+ if (entry == NULL)+ return;++ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);++ if (entry->header.backend_type == B_INVALID)+ {+ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);+ return;+ }++ /* Accumulate current stats into global before zeroing */+ LOCK_ACCUMULATE_COUNTERS(shmem->stats.stats, entry->stats.stats);++ /* Zero stats and set reset timestamp */+ memset(&entry->stats.stats, 0, sizeof(entry->stats.stats));+ entry->stats.stat_reset_timestamp = ts;- pgstat_count_backend_lock_waits(locktag_type, usecs);+ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.cindex ef541205db4..7922519f426 100644--- a/src/backend/utils/adt/pgstatfuncs.c+++ b/src/backend/utils/adt/pgstatfuncs.c@@ -1872,19 +1872,36 @@ pg_stat_get_backend_lock(PG_FUNCTION_ARGS)
{
int pid;
ReturnSetInfo *rsinfo;
- PgStat_Backend *backend_stats;+ PGPROC *proc;+ ProcNumber procnum;+ PgBackendStatus *beentry;+ PgStat_Lock *lock_stats;
InitMaterializedSRF(fcinfo, 0);
rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
pid = PG_GETARG_INT32(0);
- backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);- if (!backend_stats)+ proc = BackendPidGetProc(pid);++ if (!proc)+ proc = AuxiliaryPidGetProc(pid);+ if (!proc)
return (Datum) 0;
- pg_stat_lock_build_tuples(rsinfo, backend_stats->lock_stats.stats,- backend_stats->stat_reset_timestamp);+ procnum = GetNumberFromPGProc(proc);+ beentry = pgstat_get_beentry_by_proc_number(procnum);++ if (!beentry || beentry->st_procpid != pid)+ return (Datum) 0;++ lock_stats = pgstat_fetch_stat_backend_lock(procnum);++ if (!lock_stats)+ return (Datum) 0;++ pg_stat_lock_build_tuples(rsinfo, lock_stats->stats,+ lock_stats->stat_reset_timestamp);
return (Datum) 0;
}
@@ -2212,13 +2229,14 @@ pg_stat_reset_backend_stats(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
/*
- * Accumulate the backend's WAL stats into the global stats, then zero the- * entry.+ * Accumulate the backend's WAL and lock stats into the global stats, then+ * zero the entries.
*/
ts = GetCurrentTimestamp();
pgstat_wal_reset_backend_cb(procNumber, ts);
+ pgstat_lock_reset_backend_cb(procNumber, ts);- /* Reset IO and Lock stats still in PGSTAT_KIND_BACKEND */+ /* Reset IO stats still in PGSTAT_KIND_BACKEND */
pgstat_reset(PGSTAT_KIND_BACKEND, InvalidOid, procNumber);
PG_RETURN_VOID();
diff --git a/src/include/pgstat.h b/src/include/pgstat.hindex 9a71b8d3cd8..960f0001bd6 100644--- a/src/include/pgstat.h+++ b/src/include/pgstat.h@@ -593,7 +593,6 @@ typedef struct PgStat_Backend
{
TimestampTz stat_reset_timestamp;
PgStat_BktypeIO io_stats;
- PgStat_PendingLock lock_stats;
} PgStat_Backend;
/* ---------
@@ -606,12 +605,6 @@ typedef struct PgStat_BackendPending
* Backend statistics store the same amount of IO data as PGSTAT_KIND_IO.
*/
PgStat_PendingIO pending_io;
-- /*- * Backend statistics store the same amount of lock data as- * PGSTAT_KIND_LOCK.- */- PgStat_PendingLock pending_lock;
} PgStat_BackendPending;
/*
@@ -664,9 +657,6 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
IOOp io_op, uint32 cnt,
uint64 bytes);
-/* used by pgstat_lock.c for lock stats tracked in backends */-extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);-extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
@@ -723,6 +713,8 @@ extern void pgstat_count_lock_fastpath_exceeded(uint8 locktag_type);
extern void pgstat_count_lock_waits(uint8 locktag_type,
PgStat_Counter usecs);
extern PgStat_Lock *pgstat_fetch_stat_lock(void);
+extern PgStat_Lock *pgstat_fetch_stat_backend_lock(ProcNumber procnum);+extern void pgstat_lock_reset_backend_cb(ProcNumber procnum, TimestampTz ts);
/*
* Functions in pgstat_database.c
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.hindex d0c9379b42b..474be775430 100644--- a/src/include/utils/pgstat_internal.h+++ b/src/include/utils/pgstat_internal.h@@ -502,7 +502,10 @@ typedef struct PgStatShared_IO
typedef struct PgStatShared_Lock
{
- /* lock protects ->stats */+ /*+ * The lock protects global stats and serializes snapshots and resets with+ * transfers from per-backend entries.+ */
LWLock lock;
PgStat_Lock stats;
} PgStatShared_Lock;
@@ -532,6 +535,16 @@ typedef struct PgStatShared_PerBackendEntry
BackendType backend_type;
} PgStatShared_PerBackendEntry;
+/*+ * Per-backend entry for lock statistics, stored in an array indexed by+ * ProcNumber.+ */+typedef struct PgStatShared_LockBackendEntry+{+ PgStatShared_PerBackendEntry header;+ PgStat_Lock stats;+} PgStatShared_LockBackendEntry;+
/*
* Per-backend entry for WAL statistics, stored in an array indexed by
* ProcNumber.
@@ -811,8 +824,7 @@ extern void pgstat_archiver_snapshot_cb(void);
/* flags for pgstat_flush_backend() */
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
-#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_LOCK)+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
@@ -878,6 +890,9 @@ extern bool pgstat_lock_flush_cb(bool nowait);
extern void pgstat_lock_init_shmem_cb(void *stats);
extern void pgstat_lock_reset_all_cb(TimestampTz ts);
extern void pgstat_lock_snapshot_cb(void);
+extern void pgstat_lock_acc_backend_cb(void);+extern void pgstat_lock_acc_all_backends(void);+extern void pgstat_lock_per_backend_acc_cb(void *dst, void *entry);
/*
* Functions in pgstat_relation.c
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.listindex 95b87d32a62..549141f592d 100644--- a/src/tools/pgindent/typedefs.list+++ b/src/tools/pgindent/typedefs.list@@ -2302,6 +2302,7 @@ PgStatShared_HashEntry
PgStatShared_Index
PgStatShared_IO
PgStatShared_Lock
+PgStatShared_LockBackendEntry
PgStatShared_PerBackendEntry
PgStatShared_Relation
PgStatShared_ReplSlot
--
2.34.1
v4-0005-pgstat-move-I-O-statistics-to-new-per-backend-inf.patch (text/x-diff, 32.4K) download
From 766aafba3598dc9844dbaa4e2baff9174c6341cc Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Fri, 17 Jul 2026 13:26:07 +0000
Subject: [PATCH v4 5/6] pgstat: move I/O statistics to new per-backend
infrastructure
PGSTAT_KIND_BACKEND stores each backend's I/O statistics in one
variable-numbered entry keyed by ProcNumber. Move I/O statistics into a
dedicated fixed array associated with the I/O statistics kind and indexed
by ProcNumber. The global I/O stats hold data for backends that have exited,
while the array holds statistics for active backends.
Build pg_stat_io snapshots by copying the global stats and adding every
active per-backend entry.
Transfer the current process's entry into the global stats before
invalidating it at process exit or ProcNumber reuse. Transfer all entries
before a clean shutdown writes the statistics file.
Remove I/O counters from PGSTAT_KIND_BACKEND and use the new infrastructure
in pg_stat_get_backend_io(). This also makes I/O statistics available for
auxiliary and shared-memory-only workers. Update the documentation to
describe the resulting behavior.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
doc/src/sgml/monitoring.sgml | 9 +-
src/backend/replication/walsender.c | 2 -
src/backend/utils/activity/pgstat.c | 9 +-
src/backend/utils/activity/pgstat_backend.c | 155 +-------------
src/backend/utils/activity/pgstat_io.c | 213 ++++++++++++++-----
src/backend/utils/activity/pgstat_relation.c | 2 -
src/backend/utils/activity/pgstat_wal.c | 1 -
src/backend/utils/adt/pgstatfuncs.c | 47 ++--
src/include/pgstat.h | 37 ++--
src/include/utils/pgstat_internal.h | 25 ++-
src/test/regress/expected/stats.out | 25 ++-
src/test/regress/sql/stats.sql | 13 +-
src/tools/pgindent/typedefs.list | 3 +-
13 files changed, 255 insertions(+), 286 deletions(-)
66.9% src/backend/utils/activity/
8.7% src/backend/utils/adt/
5.2% src/include/utils/
6.0% src/include/
5.0% src/test/regress/expected/
3.9% src/test/regress/sql/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgmlindex 0602d80e599..6bb840e19cd 100644--- a/doc/src/sgml/monitoring.sgml+++ b/doc/src/sgml/monitoring.sgml@@ -5812,12 +5812,6 @@ description | Waiting for a newly initialized WAL file to reach durable storage
Returns I/O statistics about the backend with the specified
process ID. The output fields are exactly the same as the ones in the
<structname>pg_stat_io</structname> view.
- </para>- <para>- The function does not return I/O statistics for the checkpointer,- the background writer, the startup process and the autovacuum launcher- as they are already visible in the <structname>pg_stat_io</structname>- view and there is only one of each.
</para></entry>
</row>
@@ -6015,7 +6009,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage
<listitem>
<para>
<literal>io</literal>: Reset all the counters shown in the
- <structname>pg_stat_io</structname> view.+ <structname>pg_stat_io</structname> view, as well as per-backend+ I/O statistics returned by <function>pg_stat_get_backend_io</function>.
</para>
</listitem>
<listitem>
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.cindex 9f21c7c8dfe..d4ab9c26f8e 100644--- a/src/backend/replication/walsender.c+++ b/src/backend/replication/walsender.c@@ -2091,7 +2091,6 @@ WalSndWaitForWal(XLogRecPtr loc)
WALSENDER_STATS_FLUSH_INTERVAL))
{
pgstat_flush_io(false);
- (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
last_flush = now;
}
@@ -3210,7 +3209,6 @@ WalSndLoop(WalSndSendDataCallback send_data)
WALSENDER_STATS_FLUSH_INTERVAL))
{
pgstat_flush_io(false);
- (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
last_flush = now;
}
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.cindex 1affde4df05..411733e72f4 100644--- a/src/backend/utils/activity/pgstat.c+++ b/src/backend/utils/activity/pgstat.c@@ -393,7 +393,6 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.shared_data_off = offsetof(PgStatShared_Backend, stats),
.shared_data_len = sizeof(((PgStatShared_Backend *) 0)->stats),
- .flush_static_cb = pgstat_backend_flush_cb,
.reset_timestamp_cb = pgstat_backend_reset_timestamp_cb,
},
@@ -462,6 +461,11 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.init_shmem_cb = pgstat_io_init_shmem_cb,
.reset_all_cb = pgstat_io_reset_all_cb,
.snapshot_cb = pgstat_io_snapshot_cb,
++ .per_backend_entry_size = sizeof(PgStatShared_IOBackendEntry),+ .per_backend_data_off = offsetof(PgStatShared_IOBackendEntry, stats),+ .per_backend_data_len = sizeof(PgStat_BackendIO),+ .per_backend_acc_cb = pgstat_io_per_backend_acc_cb,
},
[PGSTAT_KIND_LOCK] = {
@@ -638,6 +642,7 @@ pgstat_before_server_shutdown(int code, Datum arg)
/* Transfer all live per-backend stats before writing the stats file */
pgstat_wal_acc_all_backends();
pgstat_lock_acc_all_backends();
+ pgstat_io_acc_all_backends();
pgStatLocal.shmem->is_shutdown = true;
pgstat_write_statsfile();
@@ -685,6 +690,7 @@ pgstat_shutdown_hook(int code, Datum arg)
/* Accumulate per-backend stats into the global stats */
pgstat_wal_acc_backend_cb();
pgstat_lock_acc_backend_cb();
+ pgstat_io_acc_backend_cb();
pgstat_detach_shmem();
@@ -713,6 +719,7 @@ pgstat_initialize(void)
*/
pgstat_wal_acc_backend_cb();
pgstat_lock_acc_backend_cb();
+ pgstat_io_acc_backend_cb();
/*
* Activate per-backend statistics entries here. This also covers
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.cindex 8dda2cd88c1..b5996d15d8f 100644--- a/src/backend/utils/activity/pgstat_backend.c+++ b/src/backend/utils/activity/pgstat_backend.c@@ -11,9 +11,7 @@
* This statistics kind uses a proc number as object ID for the hash table
* of pgstats. Entries are created each time a process is spawned, and are
* dropped when the process exits. These are not written to the pgstats file
- * on disk. Pending statistics are managed without direct interactions with- * PgStat_EntryRef->pending, relying on PendingBackendStats instead so as it- * is possible to report data within critical sections.+ * on disk.
*
* Copyright (c) 2001-2026, PostgreSQL Global Development Group
*
@@ -24,60 +22,10 @@
#include "postgres.h"
-#include "access/xlog.h"-#include "executor/instrument.h"-#include "storage/bufmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
-#include "utils/memutils.h"
#include "utils/pgstat_internal.h"
-/*- * Backend statistics counts waiting to be flushed out. These counters may be- * reported within critical sections so we use static memory in order to avoid- * memory allocation.- */-static PgStat_BackendPending PendingBackendStats;-static bool backend_has_iostats = false;--/*- * Utility routines to report I/O stats for backends, kept here to avoid- * exposing PendingBackendStats to the outside world.- */-void-pgstat_count_backend_io_op_time(IOObject io_object, IOContext io_context,- IOOp io_op, instr_time io_time)-{- Assert(track_io_timing || track_wal_io_timing);-- if (!pgstat_tracks_backend_bktype(MyBackendType))- return;-- Assert(pgstat_tracks_io_op(MyBackendType, io_object, io_context, io_op));-- INSTR_TIME_ADD(PendingBackendStats.pending_io.pending_times[io_object][io_context][io_op],- io_time);-- backend_has_iostats = true;- pgstat_report_fixed = true;-}--void-pgstat_count_backend_io_op(IOObject io_object, IOContext io_context,- IOOp io_op, uint32 cnt, uint64 bytes)-{- if (!pgstat_tracks_backend_bktype(MyBackendType))- return;-- Assert(pgstat_tracks_io_op(MyBackendType, io_object, io_context, io_op));-- PendingBackendStats.pending_io.counts[io_object][io_context][io_op] += cnt;- PendingBackendStats.pending_io.bytes[io_object][io_context][io_op] += bytes;-- backend_has_iostats = true;- pgstat_report_fixed = true;-}-
/*
* Returns statistics of a backend by proc number.
*/
@@ -151,104 +99,6 @@ pgstat_fetch_stat_backend_by_pid(int pid, BackendType *bktype)
return backend_stats;
}
-/*- * Flush out locally pending backend IO statistics. Locking is managed- * by the caller.- */-static void-pgstat_flush_backend_entry_io(PgStat_EntryRef *entry_ref)-{- PgStatShared_Backend *shbackendent;- PgStat_BktypeIO *bktype_shstats;- PgStat_PendingIO pending_io;-- /*- * This function can be called even if nothing at all has happened for IO- * statistics. In this case, avoid unnecessarily modifying the stats- * entry.- */- if (!backend_has_iostats)- return;-- shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;- bktype_shstats = &shbackendent->stats.io_stats;- pending_io = PendingBackendStats.pending_io;-- for (int io_object = 0; io_object < IOOBJECT_NUM_TYPES; io_object++)- {- for (int io_context = 0; io_context < IOCONTEXT_NUM_TYPES; io_context++)- {- for (int io_op = 0; io_op < IOOP_NUM_TYPES; io_op++)- {- instr_time time;-- bktype_shstats->counts[io_object][io_context][io_op] +=- pending_io.counts[io_object][io_context][io_op];- bktype_shstats->bytes[io_object][io_context][io_op] +=- pending_io.bytes[io_object][io_context][io_op];- time = pending_io.pending_times[io_object][io_context][io_op];-- bktype_shstats->times[io_object][io_context][io_op] +=- INSTR_TIME_GET_MICROSEC(time);- }- }- }-- /*- * Clear out the statistics buffer, so it can be re-used.- */- MemSet(&PendingBackendStats.pending_io, 0, sizeof(PgStat_PendingIO));-- backend_has_iostats = false;-}--/*- * Flush out locally pending backend statistics- *- * "flags" parameter controls which statistics to flush. Returns true- * if some statistics could not be flushed due to lock contention.- */-bool-pgstat_flush_backend(bool nowait, uint32 flags)-{- PgStat_EntryRef *entry_ref;- bool has_pending_data = false;-- if (!pgstat_tracks_backend_bktype(MyBackendType))- return false;-- /* Some IO data pending? */- if ((flags & PGSTAT_BACKEND_FLUSH_IO) && backend_has_iostats)- has_pending_data = true;-- if (!has_pending_data)- return false;-- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_BACKEND, InvalidOid,- MyProcNumber, nowait);- if (!entry_ref)- return true;-- /* Flush requested statistics */- if (flags & PGSTAT_BACKEND_FLUSH_IO)- pgstat_flush_backend_entry_io(entry_ref);-- pgstat_unlock_entry(entry_ref);-- return false;-}--/*- * Callback to flush out locally pending backend statistics.- *- * If some stats could not be flushed due to lock contention, return true.- */-bool-pgstat_backend_flush_cb(bool nowait)-{- return pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_ALL);-}-
/*
* Create backend statistics entry for proc number.
*/
@@ -268,9 +118,6 @@ pgstat_create_backend(ProcNumber procnum)
*/
memset(&shstatent->stats, 0, sizeof(shstatent->stats));
pgstat_unlock_entry(entry_ref);
-- MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));- backend_has_iostats = false;
}
/*
diff --git a/src/backend/utils/activity/pgstat_io.c b/src/backend/utils/activity/pgstat_io.cindex 3c8fcb20aba..85178ff1d54 100644--- a/src/backend/utils/activity/pgstat_io.c+++ b/src/backend/utils/activity/pgstat_io.c@@ -7,6 +7,11 @@
* from pgstat.c to enforce the line between the statistics access / storage
* implementation and the details about individual types of statistics.
*
+ * I/O statistics use a per-backend array to avoid double-counting. Each+ * process flushes I/O stats to its own ProcNumber-indexed entry. The global+ * pg_stat_io view aggregates the global stats (which holds stats from exited+ * processes) plus all active per-backend entries.+ *
* Copyright (c) 2021-2026, PostgreSQL Global Development Group
*
* IDENTIFICATION
@@ -68,9 +73,6 @@ pgstat_count_io_op(IOObject io_object, IOContext io_context, IOOp io_op,
PendingIOStats.counts[io_object][io_context][io_op] += cnt;
PendingIOStats.bytes[io_object][io_context][io_op] += bytes;
- /* Add the per-backend counts */- pgstat_count_backend_io_op(io_object, io_context, io_op, cnt, bytes);-
have_iostats = true;
pgstat_report_fixed = true;
}
@@ -143,10 +145,6 @@ pgstat_count_io_op_time(IOObject io_object, IOContext io_context, IOOp io_op,
INSTR_TIME_ADD(PendingIOStats.pending_times[io_object][io_context][io_op],
io_time);
-- /* Add the per-backend count */- pgstat_count_backend_io_op_time(io_object, io_context, io_op,- io_time);
}
pgstat_count_io_op(io_object, io_context, io_op, cnt, bytes);
@@ -170,7 +168,7 @@ pgstat_flush_io(bool nowait)
}
/*
- * Flush out locally pending IO statistics+ * Flush out locally pending I/O statistics to the per-backend array entry.
*
* If no stats have been recorded, this function returns false.
*
@@ -180,20 +178,18 @@ pgstat_flush_io(bool nowait)
bool
pgstat_io_flush_cb(bool nowait)
{
- LWLock *bktype_lock;+ PgStatShared_IOBackendEntry *entry;
PgStat_BktypeIO *bktype_shstats;
if (!have_iostats)
return false;
- bktype_lock = &pgStatLocal.shmem->io.locks[MyBackendType];- bktype_shstats =- &pgStatLocal.shmem->io.stats.stats[MyBackendType];+ entry = pgstat_lock_my_per_backend_entry(PGSTAT_KIND_IO, nowait);- if (!nowait)- LWLockAcquire(bktype_lock, LW_EXCLUSIVE);- else if (!LWLockConditionalAcquire(bktype_lock, LW_EXCLUSIVE))- return true;+ if (entry == NULL)+ return nowait;++ bktype_shstats = &entry->stats.stats;
for (int io_object = 0; io_object < IOOBJECT_NUM_TYPES; io_object++)
{
@@ -217,12 +213,9 @@ pgstat_io_flush_cb(bool nowait)
}
}
- Assert(pgstat_bktype_io_stats_valid(bktype_shstats, MyBackendType));-- LWLockRelease(bktype_lock);+ LWLockRelease(&entry->header.lock);
memset(&PendingIOStats, 0, sizeof(PendingIOStats));
-
have_iostats = false;
return false;
@@ -271,55 +264,63 @@ pgstat_io_init_shmem_cb(void *stats)
{
PgStatShared_IO *stat_shmem = (PgStatShared_IO *) stats;
- for (int i = 0; i < BACKEND_NUM_TYPES; i++)- LWLockInitialize(&stat_shmem->locks[i], LWTRANCHE_PGSTATS_DATA);+ LWLockInitialize(&stat_shmem->lock, LWTRANCHE_PGSTATS_DATA);
}
void
pgstat_io_reset_all_cb(TimestampTz ts)
{
- for (int i = 0; i < BACKEND_NUM_TYPES; i++)- {- LWLock *bktype_lock = &pgStatLocal.shmem->io.locks[i];- PgStat_BktypeIO *bktype_shstats = &pgStatLocal.shmem->io.stats.stats[i];+ PgStatShared_IO *shmem = &pgStatLocal.shmem->io;- LWLockAcquire(bktype_lock, LW_EXCLUSIVE);+ /*+ * Hold the kind lock while resetting both the global stats and active+ * entries. Transfers hold the same lock, so pre-reset counters cannot be+ * moved into the global stats after it is reset.+ */+ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ memset(&shmem->stats, 0, sizeof(shmem->stats));+ shmem->stats.stat_reset_timestamp = ts;- /*- * Use the lock in the first BackendType's PgStat_BktypeIO to protect- * the reset timestamp as well.- */- if (i == 0)- pgStatLocal.shmem->io.stats.stat_reset_timestamp = ts;+ /* Reset all per-backend entries */+ for (ProcNumber procnum = 0; procnum < NumPerBackendStatSlots; procnum++)+ {+ PgStatShared_IOBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_IO, procnum);- memset(bktype_shstats, 0, sizeof(*bktype_shstats));- LWLockRelease(bktype_lock);+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);+ if (entry->header.backend_type != B_INVALID)+ {+ memset(&entry->stats.stats, 0, sizeof(PgStat_BktypeIO));+ entry->stats.stat_reset_timestamp = ts;+ }+ LWLockRelease(&entry->header.lock);
}
++ LWLockRelease(&shmem->lock);
}
+/*+ * Build an I/O stats snapshot by aggregating global stats and all active+ * per-backend entries.+ */
void
pgstat_io_snapshot_cb(void)
{
- for (int i = 0; i < BACKEND_NUM_TYPES; i++)- {- LWLock *bktype_lock = &pgStatLocal.shmem->io.locks[i];- PgStat_BktypeIO *bktype_shstats = &pgStatLocal.shmem->io.stats.stats[i];- PgStat_BktypeIO *bktype_snap = &pgStatLocal.snapshot.io.stats[i];+ PgStatShared_IO *shmem = &pgStatLocal.shmem->io;+ PgStat_IO *snap = &pgStatLocal.snapshot.io;- LWLockAcquire(bktype_lock, LW_SHARED);+ /*+ * Prevent entries from moving to the global stats between copying it and+ * scanning the per-backend array.+ */+ LWLockAcquire(&shmem->lock, LW_SHARED);+ memcpy(snap, &shmem->stats, sizeof(PgStat_IO));- /*- * Use the lock in the first BackendType's PgStat_BktypeIO to protect- * the reset timestamp as well.- */- if (i == 0)- pgStatLocal.snapshot.io.stat_reset_timestamp =- pgStatLocal.shmem->io.stats.stat_reset_timestamp;+ /* Add in all active per-backend entries */+ pgstat_per_backend_snapshot(PGSTAT_KIND_IO, snap);- /* using struct assignment due to better type safety */- *bktype_snap = *bktype_shstats;- LWLockRelease(bktype_lock);- }+ LWLockRelease(&shmem->lock);
}
/*
@@ -584,3 +585,111 @@ pgstat_tracks_io_op(BackendType bktype, IOObject io_object,
return true;
}
++/*+ * Accumulate I/O counters from src into dst.+ */+static inline void+pgstat_io_accumulate_counters(PgStat_BktypeIO *dst, const PgStat_BktypeIO *src)+{+ for (int io_object = 0; io_object < IOOBJECT_NUM_TYPES; io_object++)+ {+ for (int io_context = 0; io_context < IOCONTEXT_NUM_TYPES; io_context++)+ {+ for (int io_op = 0; io_op < IOOP_NUM_TYPES; io_op++)+ {+ dst->counts[io_object][io_context][io_op] +=+ src->counts[io_object][io_context][io_op];+ dst->bytes[io_object][io_context][io_op] +=+ src->bytes[io_object][io_context][io_op];+ dst->times[io_object][io_context][io_op] +=+ src->times[io_object][io_context][io_op];+ }+ }+ }+}++/*+ * Accumulate one per-backend I/O entry into a snapshot or the global stats.+ */+void+pgstat_io_per_backend_acc_cb(void *dst, void *entry)+{+ PgStat_IO *stats = dst;+ PgStatShared_IOBackendEntry *e = (PgStatShared_IOBackendEntry *) entry;+ BackendType bktype = e->header.backend_type;++ if (bktype == B_INVALID)+ return;++ pgstat_io_accumulate_counters(&stats->stats[bktype], &e->stats.stats);+}++/*+ * Accumulate a backend's I/O stats into the global stats, then invalidate the+ * entry.+ *+ * Called at backend exit after the final flush, or when a ProcNumber is+ * being reused.+ */+void+pgstat_io_acc_backend_cb(void)+{+ pgstat_acc_my_per_backend(PGSTAT_KIND_IO, &pgStatLocal.shmem->io.lock);+}++/*+ * Accumulate all remaining per-backend I/O stats entries into the global+ * stats and invalidate them. Called at clean server shutdown to ensure all+ * flushed data is preserved in the stats file.+ */+void+pgstat_io_acc_all_backends(void)+{+ pgstat_acc_all_per_backend(PGSTAT_KIND_IO, &pgStatLocal.shmem->io.lock);+}++/*+ * Return per-backend I/O statistics for the given ProcNumber.+ */+PgStat_BackendIO *+pgstat_fetch_stat_backend_io(ProcNumber procnum)+{+ return (PgStat_BackendIO *) pgstat_fetch_per_backend(PGSTAT_KIND_IO, procnum);+}++/*+ * Reset a backend's I/O stats. Accumulate the entry's counters into the+ * global stats, then zero the stats and set the reset timestamp.+ */+void+pgstat_io_reset_backend_cb(ProcNumber procnum, TimestampTz ts)+{+ PgStatShared_IO *shmem = &pgStatLocal.shmem->io;+ PgStatShared_IOBackendEntry *entry;++ entry = pgstat_get_per_backend_entry(PGSTAT_KIND_IO, procnum);+ if (entry == NULL)+ return;++ LWLockAcquire(&shmem->lock, LW_EXCLUSIVE);+ LWLockAcquire(&entry->header.lock, LW_EXCLUSIVE);++ if (entry->header.backend_type == B_INVALID)+ {+ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);+ return;+ }++ /* Accumulate current stats into global before zeroing */+ pgstat_io_accumulate_counters(&shmem->stats.stats[entry->header.backend_type],+ &entry->stats.stats);++ /* Zero stats and set reset timestamp */+ memset(&entry->stats, 0, sizeof(entry->stats));+ entry->stats.stat_reset_timestamp = ts;++ LWLockRelease(&entry->header.lock);+ LWLockRelease(&shmem->lock);+}diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.cindex 5c70543ba88..4c3d54c350a 100644--- a/src/backend/utils/activity/pgstat_relation.c+++ b/src/backend/utils/activity/pgstat_relation.c@@ -321,7 +321,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
- (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
/*
@@ -417,7 +416,6 @@ pgstat_report_analyze(Relation rel,
/* see pgstat_report_vacuum() */
pgstat_flush_io(false);
- (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO);
}
/*
diff --git a/src/backend/utils/activity/pgstat_wal.c b/src/backend/utils/activity/pgstat_wal.cindex 8009ac38801..064f1adc518 100644--- a/src/backend/utils/activity/pgstat_wal.c+++ b/src/backend/utils/activity/pgstat_wal.c@@ -60,7 +60,6 @@ pgstat_report_wal(bool force)
/* flush IO stats */
pgstat_flush_io(nowait);
- (void) pgstat_flush_backend(nowait, PGSTAT_BACKEND_FLUSH_IO);
}
/*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.cindex 7922519f426..b4ede4be5a8 100644--- a/src/backend/utils/adt/pgstatfuncs.c+++ b/src/backend/utils/adt/pgstatfuncs.c@@ -1671,32 +1671,47 @@ Datum
pg_stat_get_backend_io(PG_FUNCTION_ARGS)
{
ReturnSetInfo *rsinfo;
- BackendType bktype;
int pid;
- PgStat_Backend *backend_stats;- PgStat_BktypeIO *bktype_stats;+ PGPROC *proc;+ ProcNumber procnum;+ PgBackendStatus *beentry;+ BackendType bktype;+ PgStat_BackendIO *backend_io;
InitMaterializedSRF(fcinfo, 0);
rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
pid = PG_GETARG_INT32(0);
- backend_stats = pgstat_fetch_stat_backend_by_pid(pid, &bktype);- if (!backend_stats)+ proc = BackendPidGetProc(pid);++ if (!proc)+ proc = AuxiliaryPidGetProc(pid);+ if (!proc)+ return (Datum) 0;++ procnum = GetNumberFromPGProc(proc);+ beentry = pgstat_get_beentry_by_proc_number(procnum);++ if (!beentry || beentry->st_procpid != pid)
return (Datum) 0;
- bktype_stats = &backend_stats->io_stats;+ bktype = beentry->st_backendType;+ backend_io = pgstat_fetch_stat_backend_io(procnum);++ if (!backend_io)+ return (Datum) 0;
/*
* In Assert builds, we can afford an extra loop through all of the
* counters (in pg_stat_io_build_tuples()), checking that only expected
* stats are non-zero, since it keeps the non-Assert code cleaner.
*/
- Assert(pgstat_bktype_io_stats_valid(bktype_stats, bktype));+ Assert(pgstat_bktype_io_stats_valid(&backend_io->stats, bktype));++ pg_stat_io_build_tuples(rsinfo, &backend_io->stats, bktype,+ backend_io->stat_reset_timestamp);- /* save tuples with data from this PgStat_BktypeIO */- pg_stat_io_build_tuples(rsinfo, bktype_stats, bktype,- backend_stats->stat_reset_timestamp);
return (Datum) 0;
}
@@ -2224,20 +2239,14 @@ pg_stat_reset_backend_stats(PG_FUNCTION_ARGS)
if (!beentry)
PG_RETURN_VOID();
- /* Check if the backend type tracks statistics */- if (!pgstat_tracks_backend_bktype(beentry->st_backendType))- PG_RETURN_VOID();-
/*
- * Accumulate the backend's WAL and lock stats into the global stats, then- * zero the entries.+ * Accumulate the backend's WAL, lock and I/O stats into the global stats,+ * then zero the entries.
*/
ts = GetCurrentTimestamp();
pgstat_wal_reset_backend_cb(procNumber, ts);
pgstat_lock_reset_backend_cb(procNumber, ts);
-- /* Reset IO stats still in PGSTAT_KIND_BACKEND */- pgstat_reset(PGSTAT_KIND_BACKEND, InvalidOid, procNumber);+ pgstat_io_reset_backend_cb(procNumber, ts);
PG_RETURN_VOID();
}
diff --git a/src/include/pgstat.h b/src/include/pgstat.hindex 960f0001bd6..bc071c2e9aa 100644--- a/src/include/pgstat.h+++ b/src/include/pgstat.h@@ -275,7 +275,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBE
typedef struct PgStat_ArchiverStats
{
@@ -390,6 +390,15 @@ typedef struct PgStat_BktypeIO
PgStat_Counter times[IOOBJECT_NUM_TYPES][IOCONTEXT_NUM_TYPES][IOOP_NUM_TYPES];
} PgStat_BktypeIO;
+/*+ * I/O statistics for one process, with an independent reset timestamp.+ */+typedef struct PgStat_BackendIO+{+ TimestampTz stat_reset_timestamp;+ PgStat_BktypeIO stats;+} PgStat_BackendIO;+
typedef struct PgStat_PendingIO
{
uint64 bytes[IOOBJECT_NUM_TYPES][IOCONTEXT_NUM_TYPES][IOOP_NUM_TYPES];
@@ -592,21 +601,8 @@ typedef struct PgStat_WalStats
typedef struct PgStat_Backend
{
TimestampTz stat_reset_timestamp;
- PgStat_BktypeIO io_stats;
} PgStat_Backend;
-/* ---------- * PgStat_BackendPending Non-flushed backend stats.- * ---------- */-typedef struct PgStat_BackendPending-{- /*- * Backend statistics store the same amount of IO data as PGSTAT_KIND_IO.- */- PgStat_PendingIO pending_io;-} PgStat_BackendPending;-
/*
* Functions in pgstat.c
*/
@@ -647,17 +643,6 @@ extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
* Functions in pgstat_backend.c
*/
-/* used by pgstat_io.c for I/O stats tracked in backends */-extern void pgstat_count_backend_io_op_time(IOObject io_object,- IOContext io_context,- IOOp io_op,- instr_time io_time);-extern void pgstat_count_backend_io_op(IOObject io_object,- IOContext io_context,- IOOp io_op, uint32 cnt,- uint64 bytes);--
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
@@ -694,6 +679,8 @@ extern void pgstat_count_io_op_time(IOObject io_object, IOContext io_context,
uint32 cnt, uint64 bytes);
extern PgStat_IO *pgstat_fetch_stat_io(void);
+extern PgStat_BackendIO *pgstat_fetch_stat_backend_io(ProcNumber procnum);+extern void pgstat_io_reset_backend_cb(ProcNumber procnum, TimestampTz ts);
extern const char *pgstat_get_io_context_name(IOContext io_context);
extern const char *pgstat_get_io_object_name(IOObject io_object);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.hindex 474be775430..393f09c477a 100644--- a/src/include/utils/pgstat_internal.h+++ b/src/include/utils/pgstat_internal.h@@ -493,10 +493,10 @@ typedef struct PgStatShared_Checkpointer
typedef struct PgStatShared_IO
{
/*
- * locks[i] protects stats.stats[i]. locks[0] also protects- * stats.stat_reset_timestamp.+ * The lock protects global stats and serializes snapshots and resets with+ * transfers from per-backend entries.
*/
- LWLock locks[BACKEND_NUM_TYPES];+ LWLock lock;
PgStat_IO stats;
} PgStatShared_IO;
@@ -535,6 +535,16 @@ typedef struct PgStatShared_PerBackendEntry
BackendType backend_type;
} PgStatShared_PerBackendEntry;
+/*+ * Per-backend entry for I/O statistics, stored in an array indexed by+ * ProcNumber.+ */+typedef struct PgStatShared_IOBackendEntry+{+ PgStatShared_PerBackendEntry header;+ PgStat_BackendIO stats;+} PgStatShared_IOBackendEntry;+
/*
* Per-backend entry for lock statistics, stored in an array indexed by
* ProcNumber.
@@ -822,12 +832,6 @@ extern void pgstat_archiver_snapshot_cb(void);
* Functions in pgstat_backend.c
*/
-/* flags for pgstat_flush_backend() */-#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO)--extern bool pgstat_flush_backend(bool nowait, uint32 flags);-extern bool pgstat_backend_flush_cb(bool nowait);
extern void pgstat_backend_reset_timestamp_cb(PgStatShared_Common *header,
TimestampTz ts);
@@ -881,6 +885,9 @@ extern bool pgstat_io_flush_cb(bool nowait);
extern void pgstat_io_init_shmem_cb(void *stats);
extern void pgstat_io_reset_all_cb(TimestampTz ts);
extern void pgstat_io_snapshot_cb(void);
+extern void pgstat_io_acc_backend_cb(void);+extern void pgstat_io_acc_all_backends(void);+extern void pgstat_io_per_backend_acc_cb(void *dst, void *entry);
/*
* Functions in pgstat_lock.c
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.outindex e3dd0ce89b0..9f1f635234d 100644--- a/src/test/regress/expected/stats.out+++ b/src/test/regress/expected/stats.out@@ -1956,14 +1956,21 @@ SELECT :io_stats_post_reset < :io_stats_pre_reset;
SELECT sum(evictions) + sum(reuses) + sum(extends) + sum(fsyncs) + sum(reads) + sum(writes) + sum(writebacks) + sum(hits) AS my_io_stats_post_reset
FROM pg_stat_get_backend_io(pg_backend_pid()) \gset
--- pg_stat_reset_shared() did not reset backend IO stats-SELECT :my_io_stats_pre_reset <= :my_io_stats_post_reset;+-- pg_stat_reset_shared() also resets per-backend IO stats+SELECT :my_io_stats_pre_reset > :my_io_stats_post_reset;
?column?
----------
t
(1 row)
--- but pg_stat_reset_backend_stats() does+SELECT bool_and(stats_reset IS NOT NULL) AS backend_io_reset_timestamp_set+ FROM pg_stat_get_backend_io(pg_backend_pid());+ backend_io_reset_timestamp_set +--------------------------------+ t+(1 row)++-- pg_stat_reset_backend_stats() also resets per-backend IO stats
SELECT pg_stat_reset_backend_stats(pg_backend_pid());
pg_stat_reset_backend_stats
-----------------------------
@@ -1989,11 +1996,13 @@ SELECT pg_stat_get_backend_io(0);
------------------------
(0 rows)
--- Auxiliary processes return no data.-SELECT pg_stat_get_backend_io(:checkpointer_pid);- pg_stat_get_backend_io --------------------------(0 rows)+-- Auxiliary processes now return data+SELECT count(*) > 0 AS checkpointer_has_io_stats+ FROM pg_stat_get_backend_io(:checkpointer_pid);+ checkpointer_has_io_stats +---------------------------+ t+(1 row)
-- test BRIN index doesn't block HOT update
CREATE TABLE brin_hot (
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sqlindex f4931fd84de..3f88c04f0aa 100644--- a/src/test/regress/sql/stats.sql+++ b/src/test/regress/sql/stats.sql@@ -924,9 +924,11 @@ SELECT sum(evictions) + sum(reuses) + sum(extends) + sum(fsyncs) + sum(reads) +
SELECT :io_stats_post_reset < :io_stats_pre_reset;
SELECT sum(evictions) + sum(reuses) + sum(extends) + sum(fsyncs) + sum(reads) + sum(writes) + sum(writebacks) + sum(hits) AS my_io_stats_post_reset
FROM pg_stat_get_backend_io(pg_backend_pid()) \gset
--- pg_stat_reset_shared() did not reset backend IO stats-SELECT :my_io_stats_pre_reset <= :my_io_stats_post_reset;--- but pg_stat_reset_backend_stats() does+-- pg_stat_reset_shared() also resets per-backend IO stats+SELECT :my_io_stats_pre_reset > :my_io_stats_post_reset;+SELECT bool_and(stats_reset IS NOT NULL) AS backend_io_reset_timestamp_set+ FROM pg_stat_get_backend_io(pg_backend_pid());+-- pg_stat_reset_backend_stats() also resets per-backend IO stats
SELECT pg_stat_reset_backend_stats(pg_backend_pid());
SELECT sum(evictions) + sum(reuses) + sum(extends) + sum(fsyncs) + sum(reads) + sum(writes) + sum(writebacks) + sum(hits) AS my_io_stats_post_backend_reset
FROM pg_stat_get_backend_io(pg_backend_pid()) \gset
@@ -935,8 +937,9 @@ SELECT :my_io_stats_pre_reset > :my_io_stats_post_backend_reset;
-- Check invalid input for pg_stat_get_backend_io()
SELECT pg_stat_get_backend_io(NULL);
SELECT pg_stat_get_backend_io(0);
--- Auxiliary processes return no data.-SELECT pg_stat_get_backend_io(:checkpointer_pid);+-- Auxiliary processes now return data+SELECT count(*) > 0 AS checkpointer_has_io_stats+ FROM pg_stat_get_backend_io(:checkpointer_pid);
-- test BRIN index doesn't block HOT update
CREATE TABLE brin_hot (
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.listindex 549141f592d..f5e559945f2 100644--- a/src/tools/pgindent/typedefs.list+++ b/src/tools/pgindent/typedefs.list@@ -2301,6 +2301,7 @@ PgStatShared_Function
PgStatShared_HashEntry
PgStatShared_Index
PgStatShared_IO
+PgStatShared_IOBackendEntry
PgStatShared_Lock
PgStatShared_LockBackendEntry
PgStatShared_PerBackendEntry
@@ -2312,7 +2313,7 @@ PgStatShared_Wal
PgStatShared_WalBackendEntry
PgStat_ArchiverStats
PgStat_Backend
-PgStat_BackendPending+PgStat_BackendIO
PgStat_BackendSubEntry
PgStat_BgWriterStats
PgStat_BktypeIO
--
2.34.1
v4-0006-pgstat-remove-PGSTAT_KIND_BACKEND.patch (text/x-diff, 13.9K) download
From d1e680aac369caae67463e1001b3b136410796f8 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Thu, 13 Aug 2026 12:28:55 +0000
Subject: [PATCH v4 6/6] pgstat: remove PGSTAT_KIND_BACKEND
After WAL, Lock, and IO have moved, PGSTAT_KIND_BACKEND contains no data.
Remove that kind and pgstat_backend.c together with their build entries,
shared structures, callbacks, and call sites.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/anCjoFkGah/uB7lw%40bdtpg
---
src/backend/utils/activity/Makefile | 1 -
src/backend/utils/activity/backend_status.c | 4 -
src/backend/utils/activity/meson.build | 1 -
src/backend/utils/activity/pgstat.c | 20 ---
src/backend/utils/activity/pgstat_backend.c | 179 --------------------
src/include/pgstat.h | 19 ---
src/include/utils/pgstat_internal.h | 13 --
src/include/utils/pgstat_kind.h | 15 +-
src/test/regress/expected/stats.out | 17 +-
src/tools/pgindent/typedefs.list | 2 -
10 files changed, 15 insertions(+), 256 deletions(-)
72.8% src/backend/utils/activity/
9.8% src/include/utils/
6.4% src/include/
10.3% src/test/regress/expected/
diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefileindex 8924b10ed2e..618a04602c8 100644--- a/src/backend/utils/activity/Makefile+++ b/src/backend/utils/activity/Makefile@@ -20,7 +20,6 @@ OBJS = \
backend_status.o \
pgstat.o \
pgstat_archiver.o \
- pgstat_backend.o \
pgstat_bgwriter.o \
pgstat_checkpointer.o \
pgstat_database.o \
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.cindex d685fc5cd87..7088fc09651 100644--- a/src/backend/utils/activity/backend_status.c+++ b/src/backend/utils/activity/backend_status.c@@ -458,10 +458,6 @@ pgstat_bestart_final(void)
PGSTAT_END_WRITE_ACTIVITY(beentry);
- /* Create the backend statistics entry */- if (pgstat_tracks_backend_bktype(MyBackendType))- pgstat_create_backend(MyProcNumber);-
/* Update app name to current GUC setting */
if (application_name)
pgstat_report_appname(application_name);
diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.buildindex 6434c377e53..ea6495b95d3 100644--- a/src/backend/utils/activity/meson.build+++ b/src/backend/utils/activity/meson.build@@ -5,7 +5,6 @@ backend_sources += files(
'backend_status.c',
'pgstat.c',
'pgstat_archiver.c',
- 'pgstat_backend.c',
'pgstat_bgwriter.c',
'pgstat_checkpointer.c',
'pgstat_database.c',
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.cindex 411733e72f4..5bb580a895b 100644--- a/src/backend/utils/activity/pgstat.c+++ b/src/backend/utils/activity/pgstat.c@@ -77,7 +77,6 @@
*
* Each statistics kind is handled in a dedicated file:
* - pgstat_archiver.c
- * - pgstat_backend.c
* - pgstat_bgwriter.c
* - pgstat_checkpointer.c
* - pgstat_database.c
@@ -381,21 +380,6 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE]
.reset_timestamp_cb = pgstat_subscription_reset_timestamp_cb,
},
- [PGSTAT_KIND_BACKEND] = {- .name = "backend",-- .fixed_amount = false,- .write_to_file = false,-- .accessed_across_databases = true,-- .shared_size = sizeof(PgStatShared_Backend),- .shared_data_off = offsetof(PgStatShared_Backend, stats),- .shared_data_len = sizeof(((PgStatShared_Backend *) 0)->stats),-- .reset_timestamp_cb = pgstat_backend_reset_timestamp_cb,- },-
/* stats for fixed-numbered (mostly 1) objects */
[PGSTAT_KIND_ARCHIVER] = {
@@ -683,10 +667,6 @@ pgstat_shutdown_hook(int code, Datum arg)
Assert(dlist_is_empty(&pgStatPending));
dlist_init(&pgStatPending);
- /* drop the backend stats entry */- if (!pgstat_drop_entry(PGSTAT_KIND_BACKEND, InvalidOid, MyProcNumber, false))- pgstat_request_entry_refs_gc();-
/* Accumulate per-backend stats into the global stats */
pgstat_wal_acc_backend_cb();
pgstat_lock_acc_backend_cb();
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.cdeleted file mode 100644index b5996d15d8f..00000000000--- a/src/backend/utils/activity/pgstat_backend.c+++ /dev/null@@ -1,179 +0,0 @@-/* -------------------------------------------------------------------------- *- * pgstat_backend.c- * Implementation of backend statistics.- *- * This file contains the implementation of backend statistics. It is kept- * separate from pgstat.c to enforce the line between the statistics access /- * storage implementation and the details about individual types of- * statistics.- *- * This statistics kind uses a proc number as object ID for the hash table- * of pgstats. Entries are created each time a process is spawned, and are- * dropped when the process exits. These are not written to the pgstats file- * on disk.- *- * Copyright (c) 2001-2026, PostgreSQL Global Development Group- *- * IDENTIFICATION- * src/backend/utils/activity/pgstat_backend.c- * -------------------------------------------------------------------------- */--#include "postgres.h"--#include "storage/proc.h"-#include "storage/procarray.h"-#include "utils/pgstat_internal.h"--/*- * Returns statistics of a backend by proc number.- */-PgStat_Backend *-pgstat_fetch_stat_backend(ProcNumber procNumber)-{- PgStat_Backend *backend_entry;-- backend_entry = (PgStat_Backend *) pgstat_fetch_entry(PGSTAT_KIND_BACKEND,- InvalidOid, procNumber,- NULL);-- return backend_entry;-}--/*- * Returns statistics of a backend by pid.- *- * This routine includes sanity checks to ensure that the backend exists and- * is running. "bktype" can be optionally defined to return the BackendType- * of the backend whose statistics are returned.- */-PgStat_Backend *-pgstat_fetch_stat_backend_by_pid(int pid, BackendType *bktype)-{- PGPROC *proc;- PgBackendStatus *beentry;- ProcNumber procNumber;- PgStat_Backend *backend_stats;-- proc = BackendPidGetProc(pid);- if (bktype)- *bktype = B_INVALID;-- /* this could be an auxiliary process */- if (!proc)- proc = AuxiliaryPidGetProc(pid);-- if (!proc)- return NULL;-- procNumber = GetNumberFromPGProc(proc);-- beentry = pgstat_get_beentry_by_proc_number(procNumber);- if (!beentry)- return NULL;-- /* check if the backend type tracks statistics */- if (!pgstat_tracks_backend_bktype(beentry->st_backendType))- return NULL;-- /* if PID does not match, leave */- if (beentry->st_procpid != pid)- return NULL;-- if (bktype)- *bktype = beentry->st_backendType;-- /*- * Retrieve the entry. Note that "beentry" may be freed depending on the- * value of stats_fetch_consistency, so do not access it from this point.- */- backend_stats = pgstat_fetch_stat_backend(procNumber);- if (!backend_stats)- {- if (bktype)- *bktype = B_INVALID;- return NULL;- }-- return backend_stats;-}--/*- * Create backend statistics entry for proc number.- */-void-pgstat_create_backend(ProcNumber procnum)-{- PgStat_EntryRef *entry_ref;- PgStatShared_Backend *shstatent;-- entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_BACKEND, InvalidOid,- procnum, false);- shstatent = (PgStatShared_Backend *) entry_ref->shared_stats;-- /*- * NB: need to accept that there might be stats from an older backend,- * e.g. if we previously used this proc number.- */- memset(&shstatent->stats, 0, sizeof(shstatent->stats));- pgstat_unlock_entry(entry_ref);-}--/*- * Backend statistics are not collected for all BackendTypes.- *- * The following BackendTypes do not participate in the backend stats- * subsystem:- * - The same and for the same reasons as in pgstat_tracks_io_bktype().- * - B_BG_WRITER, B_CHECKPOINTER, B_STARTUP and B_AUTOVAC_LAUNCHER because their- * I/O stats are already visible in pg_stat_io and there is only one of those.- *- * Function returns true if BackendType participates in the backend stats- * subsystem and false if it does not.- *- * When adding a new BackendType, also consider adding relevant restrictions to- * pgstat_tracks_io_object() and pgstat_tracks_io_op().- */-bool-pgstat_tracks_backend_bktype(BackendType bktype)-{- /*- * List every type so that new backend types trigger a warning about- * needing to adjust this switch.- */- switch (bktype)- {- case B_INVALID:- case B_AUTOVAC_LAUNCHER:- case B_DEAD_END_BACKEND:- case B_ARCHIVER:- case B_LOGGER:- case B_BG_WRITER:- case B_CHECKPOINTER:- case B_IO_WORKER:- case B_STARTUP:- case B_DATACHECKSUMSWORKER_LAUNCHER:- case B_DATACHECKSUMSWORKER_WORKER:- return false;-- case B_AUTOVAC_WORKER:- case B_BACKEND:- case B_BG_WORKER:- case B_STANDALONE_BACKEND:- case B_SLOTSYNC_WORKER:- case B_WAL_RECEIVER:- case B_WAL_SENDER:- case B_WAL_SUMMARIZER:- case B_WAL_WRITER:- return true;- }-- return false;-}--void-pgstat_backend_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts)-{- ((PgStatShared_Backend *) header)->stats.stat_reset_timestamp = ts;-}diff --git a/src/include/pgstat.h b/src/include/pgstat.hindex bc071c2e9aa..6e640ab3cda 100644--- a/src/include/pgstat.h+++ b/src/include/pgstat.h@@ -594,15 +594,6 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
-/* -------- * PgStat_Backend Backend statistics- * -------- */-typedef struct PgStat_Backend-{- TimestampTz stat_reset_timestamp;-} PgStat_Backend;-
/*
* Functions in pgstat.c
*/
@@ -639,16 +630,6 @@ extern bool pgstat_have_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_report_archiver(const char *xlog, bool failed);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
-/*- * Functions in pgstat_backend.c- */--extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);-extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,- BackendType *bktype);-extern bool pgstat_tracks_backend_bktype(BackendType bktype);-extern void pgstat_create_backend(ProcNumber procnum);-
/*
* Functions in pgstat_bgwriter.c
*/
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.hindex 393f09c477a..6f38a38c22c 100644--- a/src/include/utils/pgstat_internal.h+++ b/src/include/utils/pgstat_internal.h@@ -621,12 +621,6 @@ typedef struct PgStatShared_ReplSlot
PgStat_StatReplSlotEntry stats;
} PgStatShared_ReplSlot;
-typedef struct PgStatShared_Backend-{- PgStatShared_Common header;- PgStat_Backend stats;-} PgStatShared_Backend;-
/*
* Central shared memory entry for the cumulative stats system.
*
@@ -828,13 +822,6 @@ extern void pgstat_archiver_init_shmem_cb(void *stats);
extern void pgstat_archiver_reset_all_cb(TimestampTz ts);
extern void pgstat_archiver_snapshot_cb(void);
-/*- * Functions in pgstat_backend.c- */--extern void pgstat_backend_reset_timestamp_cb(PgStatShared_Common *header,- TimestampTz ts);-
/*
* Functions in pgstat_bgwriter.c
*/
diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.hindex 45ca599d0dd..489e8fa38bf 100644--- a/src/include/utils/pgstat_kind.h+++ b/src/include/utils/pgstat_kind.h@@ -30,16 +30,15 @@
#define PGSTAT_KIND_FUNCTION 4 /* per-function statistics */
#define PGSTAT_KIND_REPLSLOT 5 /* per-slot statistics */
#define PGSTAT_KIND_SUBSCRIPTION 6 /* per-subscription statistics */
-#define PGSTAT_KIND_BACKEND 7 /* per-backend statistics */
/* stats for fixed-numbered objects */
-#define PGSTAT_KIND_ARCHIVER 8-#define PGSTAT_KIND_BGWRITER 9-#define PGSTAT_KIND_CHECKPOINTER 10-#define PGSTAT_KIND_IO 11-#define PGSTAT_KIND_LOCK 12-#define PGSTAT_KIND_SLRU 13-#define PGSTAT_KIND_WAL 14+#define PGSTAT_KIND_ARCHIVER 7+#define PGSTAT_KIND_BGWRITER 8+#define PGSTAT_KIND_CHECKPOINTER 9+#define PGSTAT_KIND_IO 10+#define PGSTAT_KIND_LOCK 11+#define PGSTAT_KIND_SLRU 12+#define PGSTAT_KIND_WAL 13
#define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE
#define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.outindex 9f1f635234d..4d9e6dd57d3 100644--- a/src/test/regress/expected/stats.out+++ b/src/test/regress/expected/stats.out@@ -120,15 +120,14 @@ SELECT id, name, fixed_amount,
4 | function | f | f | t
5 | replslot | f | t | t
6 | subscription | f | t | t
- 7 | backend | f | t | f- 8 | archiver | t | f | t- 9 | bgwriter | t | f | t- 10 | checkpointer | t | f | t- 11 | io | t | f | t- 12 | lock | t | f | t- 13 | slru | t | f | t- 14 | wal | t | f | t-(14 rows)+ 7 | archiver | t | f | t+ 8 | bgwriter | t | f | t+ 9 | checkpointer | t | f | t+ 10 | io | t | f | t+ 11 | lock | t | f | t+ 12 | slru | t | f | t+ 13 | wal | t | f | t+(13 rows)
-- ensure that both seqscan and indexscan plans are allowed
SET enable_seqscan TO on;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.listindex f5e559945f2..574cb63f26d 100644--- a/src/tools/pgindent/typedefs.list+++ b/src/tools/pgindent/typedefs.list@@ -2290,7 +2290,6 @@ PgFdwSamplingMethod
PgFdwScanState
PgIfAddrCallback
PgStatShared_Archiver
-PgStatShared_Backend
PgStatShared_BgWriter
PgStatShared_Checkpointer
PgStatShared_Common
@@ -2312,7 +2311,6 @@ PgStatShared_Subscription
PgStatShared_Wal
PgStatShared_WalBackendEntry
PgStat_ArchiverStats
-PgStat_Backend
PgStat_BackendIO
PgStat_BackendSubEntry
PgStat_BgWriterStats
--
2.34.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: bertranddrouvot.pg@gmail.com, michael@paquier.xyz, andres@anarazel.de, samimseih@gmail.com
Subject: Re: Redesign per-backend statistics
In-Reply-To: <at8/ezwmjHCWRmh2@bdtpg>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox