public inbox for [email protected]  

help / color / mirror / Atom feed

From: Andres Freund <[email protected]>
To: Michael Paquier <[email protected]>
Cc: Bertrand Drouvot <[email protected]>
Cc: [email protected]
Subject: Re: Add per-backend AIO statistics
Date: Wed, 8 Jul 2026 14:08:00 -0400
Message-ID: <et272fdhdx6yphlgzvrgsf7bgwnf3vqciwp4gxqubro42uaflp@ohslaocvwgvi> (raw)
In-Reply-To: <[email protected]>
References: <akzcq9jtWK5C92g1@bdtpg>
	<[email protected]>

Hi,

On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful, one thing that sounds very
> important to me is how this impacts workloads by default.


> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step.  Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.

I think the overhead might be ok, but I am rather doubtful that all of this
information is actually useful. You're adding quite a few counters for each
IO, do we actually need that?

E.g. what do we gain from counting:
- started (if you want to see the number of IOs that are in progress,
  cumulative stats are the wrong tool)
- executed_async (that's just the number of IOs minus executed_sync)
- completed_self (that's just the number of IOs minus executed_other)

Separately, I'm doubtful it makes sense to have only per-backend stats for
this. I think you'd almost always want the stats for exited backend
(e.g. parallel workers) too.


Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
architectural direction. It'll just end up implementing all kinds of stats,
since we'll incrementally want more and more per-backend stats.  I think what
we'd want is rather something where for each applicable stats kind we have a
shared counter for all exited backends and then per-backend counters for live
backends, with helpers to aggregate the exited + live stats to a total.

Greetings,

Andres Freund

view thread (18+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Add per-backend AIO statistics
  In-Reply-To: <et272fdhdx6yphlgzvrgsf7bgwnf3vqciwp4gxqubro42uaflp@ohslaocvwgvi>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox