Feedback-ID: id4a34324:Fastmail
Date: Wed, 8 Jul 2026 14:08:00 -0400
From: Andres Freund <andres@anarazel.de>
To: Michael Paquier <michael@paquier.xyz>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: Add per-backend AIO statistics
Message-ID: <et272fdhdx6yphlgzvrgsf7bgwnf3vqciwp4gxqubro42uaflp@ohslaocvwgvi>
References: <akzcq9jtWK5C92g1@bdtpg>
 <ak3zpA2HHh5CxQK8@paquier.xyz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ak3zpA2HHh5CxQK8@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/et272fdhdx6yphlgzvrgsf7bgwnf3vqciwp4gxqubro42uaflp%40ohslaocvwgvi>
Precedence: bulk

Hi,

On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful, one thing that sounds very
> important to me is how this impacts workloads by default.


> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step.  Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.

I think the overhead might be ok, but I am rather doubtful that all of this
information is actually useful. You're adding quite a few counters for each
IO, do we actually need that?

E.g. what do we gain from counting:
- started (if you want to see the number of IOs that are in progress,
  cumulative stats are the wrong tool)
- executed_async (that's just the number of IOs minus executed_sync)
- completed_self (that's just the number of IOs minus executed_other)

Separately, I'm doubtful it makes sense to have only per-backend stats for
this. I think you'd almost always want the stats for exited backend
(e.g. parallel workers) too.


Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
architectural direction. It'll just end up implementing all kinds of stats,
since we'll incrementally want more and more per-backend stats.  I think what
we'd want is rather something where for each applicable stats kind we have a
shared counter for all exited backends and then per-backend counters for live
backends, with helpers to aggregate the exited + live stats to a total.

Greetings,

Andres Freund