From: Andres Freund <andres@anarazel.de>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: pgstat: Infrastructure for more detailed IO statistics
Date: Thu, 09 Feb 2023 05:00:02 +0000
Message-ID: <E1pPz2Q-000BS8-5e@gemulon.postgresql.org> (raw)
pgstat: Infrastructure for more detailed IO statistics
This commit adds the infrastructure for more detailed IO statistics. The calls
to actually count IOs, a system view to access the new statistics,
documentation and tests will be added in subsequent commits, to make review
easier.
While we already had some IO statistics, e.g. in pg_stat_bgwriter and
pg_stat_database, they did not provide sufficient detail to understand what
the main sources of IO are, or whether configuration changes could avoid
IO. E.g., pg_stat_bgwriter.buffers_backend does contain the number of buffers
written out by a backend, but as that includes extending relations (always
done by backends) and writes triggered by the use of buffer access strategies,
it cannot easily be used to tune background writer or checkpointer. Similarly,
pg_stat_database.blks_read cannot easily be used to tune shared_buffers /
compute a cache hit ratio, as the use of buffer access strategies will often
prevent a large fraction of the read blocks to end up in shared_buffers.
The new IO statistics count IO operations (evict, extend, fsync, read, reuse,
and write), and are aggregated for each combination of backend type (backend,
autovacuum worker, bgwriter, etc), target object of the IO (relations, temp
relations) and context of the IO (normal, vacuum, bulkread, bulkwrite).
What is tracked in this series of patches, is sufficient to perform the
aforementioned analyses. Further details, e.g. tracking the number of buffer
hits, would make that even easier, but was left out for now, to keep the scope
of the already large patchset manageable.
Bumps PGSTAT_FILE_FORMAT_ID.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20200124195226.lth52iydq2n2uilq@alap3.anarazel.de
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/28e626bde00efe8051fbd677227c97b1aa3c6a1a
Modified Files
--------------
doc/src/sgml/monitoring.sgml | 2 +
src/backend/utils/activity/Makefile | 1 +
src/backend/utils/activity/meson.build | 1 +
src/backend/utils/activity/pgstat.c | 26 ++
src/backend/utils/activity/pgstat_bgwriter.c | 7 +-
src/backend/utils/activity/pgstat_checkpointer.c | 7 +-
src/backend/utils/activity/pgstat_io.c | 391 +++++++++++++++++++++++
src/backend/utils/activity/pgstat_relation.c | 15 +-
src/backend/utils/activity/pgstat_shmem.c | 4 +
src/backend/utils/activity/pgstat_wal.c | 4 +-
src/backend/utils/adt/pgstatfuncs.c | 11 +-
src/include/miscadmin.h | 2 +
src/include/pgstat.h | 70 +++-
src/include/utils/pgstat_internal.h | 30 ++
src/tools/pgindent/typedefs.list | 6 +
15 files changed, 569 insertions(+), 8 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: andres@anarazel.de
Subject: Re: pgsql: pgstat: Infrastructure for more detailed IO statistics
In-Reply-To: <E1pPz2Q-000BS8-5e@gemulon.postgresql.org>
References: <E1pPz2Q-000BS8-5e@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox