pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Andres Freund <andres@anarazel.de>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: pgstat: scaffolding for transactional stats creation / drop.
Date: Thu, 07 Apr 2022 02:07:46 +0000
Message-ID: <E1ncHYo-000fJs-AN@gemulon.postgresql.org> (raw)

pgstat: scaffolding for transactional stats creation / drop.

One problematic part of the current statistics collector design is that there
is no reliable way of getting rid of statistics entries. Because of that
pgstat_vacuum_stat() (called by [auto-]vacuum) matches all stats for the
current database with the catalog contents and tries to drop now-superfluous
entries. That's quite expensive. What's worse, it doesn't work on physical
replicas, despite physical replicas collection statistics entries.

This commit introduces infrastructure to create / drop statistics entries
transactionally, together with the underlying catalog objects (functions,
relations, subscriptions). pgstat_xact.c maintains a list of stats entries
created / dropped transactionally in the current transaction. To ensure the
removal of statistics entries is durable dropped statistics entries are
included in commit / abort (and prepare) records, which also ensures that
stats entries are dropped on standbys.

Statistics entries created separately from creating the underlying catalog
object (e.g. when stats were previously lost due to an immediate restart)
are *not* WAL logged. However that can only happen outside of the transaction
creating the catalog object, so it does not lead to "leaked" statistics
entries.

For this to work, functions creating / dropping functions / relations /
subscriptions need to call into pgstat. For subscriptions this was already
done when dropping subscriptions, via pgstat_report_subscription_drop() (now
renamed to pgstat_drop_subscription()).

This commit does not actually drop stats yet, it just provides the
infrastructure. It is however a largely independent piece of infrastructure,
so committing it separately makes sense.

Bumps XLOG_PAGE_MAGIC.

Author: Andres Freund <andres@anarazel.de>
Reviewed-By: Thomas Munro <thomas.munro@gmail.com>
Reviewed-By: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20220303021600.hs34ghqcw6zcokdh@alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/8b1dccd37c71ed2ff016294d8f9053a32b02b19e

Modified Files
--------------
src/backend/access/rmgrdesc/xactdesc.c           |  52 ++++++
src/backend/access/transam/twophase.c            |  45 +++++
src/backend/access/transam/xact.c                |  68 ++++++-
src/backend/catalog/heap.c                       |   7 +
src/backend/catalog/pg_proc.c                    |   5 +
src/backend/commands/functioncmds.c              |   2 +
src/backend/commands/subscriptioncmds.c          |   4 +-
src/backend/postmaster/pgstat.c                  |   2 +-
src/backend/storage/smgr/smgr.c                  |   5 -
src/backend/utils/activity/pgstat_function.c     |  22 +++
src/backend/utils/activity/pgstat_relation.c     |  37 ++--
src/backend/utils/activity/pgstat_subscription.c |  19 +-
src/backend/utils/activity/pgstat_xact.c         | 223 +++++++++++++++++++++++
src/include/access/xact.h                        |  40 +++-
src/include/access/xlog_internal.h               |   2 +-
src/include/pgstat.h                             |  11 +-
src/include/utils/pgstat_internal.h              |  13 ++
src/tools/pgindent/typedefs.list                 |   3 +
18 files changed, 524 insertions(+), 36 deletions(-)



Message-ID: <E1ncHYo-000fJs-AN@gemulon.postgresql.org>
Permalink:  ../E1ncHYo-000fJs-AN@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1ncHYo-000fJs-AN@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: andres@anarazel.de
  Subject: Re: pgsql: pgstat: scaffolding for transactional stats creation / drop.
  In-Reply-To: <E1ncHYo-000fJs-AN@gemulon.postgresql.org>
  References: <E1ncHYo-000fJs-AN@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox